John Battelle's Searchblog

Danny was kind enough to ping me about this story, which breaks the news about Google's new "Dashboard," which is, in essence, a first start toward realizing the "privacy dashboard" I asked for so long ago (and again here), back when I was posting ideas like a madman (I'm going to be doing that again shortly, so watch out...).

It's a big deal I think, even if most of us never use it. And it's very smart of Google to lead here. It really had no choice, when you think about it. And it's kind of cool to see stuff I wrote about here over three years ago happen in the real world.

There's much to say about Twitter's slow to roll out but much discussed Lists feature. I'm a fan of it, in short, for many reasons. Lists is a pretty simple idea - it lets anyone make and share a list of folks on Twitter. But it's also a powerful new signal that will help Twitter solve two of its most vexing problems - first, that of discovery, and second, that of authority. Not to mention it gives everyone a chance to add value above the level of a single "follower", more on that later.

In short, if done right, Lists will provide the Twitter ecosystem a third dimension that might just propel it beyond the hype curve and into a long term platform play. Combined (intelligently) with the new traffic coming from Google and Bing, and this could mean Very Big Things for  Twitter.

All this bears further discussion. And I promise to to that, soon. I just wanted to leave a note here that I think this is important, and hopefully, when I stop traveling and start thinking a bit more, I'll dig in here.

I'm so focused on Web 2 and FM right now I can't grok this. But you guys can - Aardvark, which I have written about in the past and will present at Web 2 next week, has launched a social search function based on their growing people-powered network. What do you all think?

Just got an email from an UBS analyst with an overview of search share data from Comscore for Sept. Summary:

Google increases Month/Month search share

Comscore’s September US search data suggests Google took some share lost over the last few months, which gives us confidence in our Q3 paid click growth ests. (12% Y/Y worldwide). Bing’s share of US search also increased, though share gains were lower than previous months as September is typically a relatively weaker month for travel, a vertical in which Microsoft has tried to differentiate itself.

Google and Microsoft took share at Yahoo!’s expense

Google’s share of US core searches in Sep was up 27 bps M/M to 64.9%, almost reaching levels prior to the launch of Bing. Bing had 9.4% of US search share (+18bps M/M) while AOL and Ask were roughly flat M/M at 3% and 3.9% respectively. Yahoo! lost 50bps M/M, resulting in 18.8% share of core US searches.

Google and Microsoft’s Y/Y query growth still strong in September

Bing’s Sep. Y/Y search volume growth was +31% Y/Y from 32% Y/Y in Aug, while Google’s search volume grew 21% Y/Y, roughly flat from 22% Y/Y growth in Aug. Yahoo!’s search volume grew 9% Y/Y from 17% Y/Y growth in Aug.

If ever there was a strong meme in search, it's the impact of social: Everyone is talking about how Facebook and Twitter are threatening Google for what I've called the "oxygen" of the web: distribution of attention.  

A little background. Google rose to prominence as the absolute winner in the Internet's distribution game. The de facto interface for knowledge navigation, Google brought signal to the noise of Web 1.0: Sure, nearly everything worth publishing was now on the web, but how on earth could you find that ONE thing that mattered to your query, NOW?

A hundred billion plus dollar business ensued: we all now use Google to find that which we want to find on the web. In particular, Google is great at delivering authority on the web for those things that had already been published and ranked: In a way, Google has become the reference librarian of the web.

But...just searching a reference library is one thing. What about finding things people are talking about right now? And wouldn't it be great if you could cross index that reference library with your social graph, so that people you trusted helped you go from query to decision?

Twitter and Facebook promise that next step in search. Let's tease this out a bit.

We have different modes when we search. Sometimes we are looking for that perfect reference point - an article on how to train a dog, for example, or a how to guide on building a treehouse. But then we hit a critical inflection - we want to validate our reference material with a real live human connection. And Google can't really do that. In short, we want to cross reference what we've learned with the experience of someone we trust.

Before the rise and ubiquity of social networks, the ability to do this was pretty serendipitous - sometimes in our reference search we found humans with whom we could connect and then learn (this happened to me in 1995 as I was searching online for my birth mother, but that's another story).

But it's happening more and more online now, thanks to our ability to use Twitter and Facebook to query our social graph. Through status updates or tweets, we can ask real people that which before we asked Google. And, by reading through the lifestreams of our network, we can discover that which we might never has asked, but nevertheless find interesting.

It's late and I'm working way too many hours to do this line of thinking justice. But I will simply state it this way: Facebook and Twitter, you need to get better at mixing traditional web search with what you've already got. And Google/Microsoft, well, vicey versa. You need to get better at mixing social into your traditional web search.

Whoever does it best, wins.

Update: A new study on the interplay of search and social media can be found here.

Thanks to the BingTweets program, I've been asked to opine on search and decision engines. I'm kind of proud of my third and final post, which riffs on the first two and goes a bit, well, meta. I'd love to know what you guys think of it. I'll repost the first half here, and link back to the whole post on the original site that commissioned the work.  

Over the past two posts I’ve outlined my hopes and frustrations around search and decision making, using my desire to acquire a classic car as an example of both the opportunity and the limitations of web search as it stands today. As an astute commentator noted on my last post – “normally a 30 minute conversation is a whole lot better for any kind of complex question.”

Which leads me to my last post in this series. What is it about a conversation? Why can we, in 30 minutes or less, boil down what otherwise might be a multi-day quest into an answer that addresses nearly all our concerns? And what might that process teach us about what the Web lacks today and might bring us tomorrow?

Well the answer is at once simple and maddenly complex. Our ability to communicate using language is the result of millions of years of physical and cultural evolution, capped off by 15-25 years of personal childhood and early adult experience. But it comes so naturally, we forget how extraordinary this simple act really is.

I once asked Larry Page, co-founder of Google, what his dream search engine looked like. His answer: The computer from Star Trek – a omnipresent, all knowing machine with which you could converse. We’re a long way from that – and when we do get there, we’re bound to arrive a with a fair amount of trepidation – after all, every major summer blockbuster seems to burst with the narrative of machines that out think humans (Matrix, Terminator, Battlestar Galactica, 2001, I Robot…you get the picture).

But I have hope. Given this is my last post in the series, allow me to wax a bit philosophical. While we in the search and Internet industry focus almost exclusively on leveraging technology to get to better answers, perhaps we might take another approach. Perhaps instead of scaling machines to the point of where they can have a “human” conversation with us (a la Turing), perhaps instead (or, as well), we might leverage machines to help connect us to just the right human with whom we might have that conversation?

Continued...

Comscore's monthly ratings are out and Bing continues a slow but steady gain in share, to the slight expense of Google and Yahoo. Bing has a massive marketing push on right now, but also, I think the service is starting to gain footholds with users who see it as a regular alternative to Google. I am also a fan of the recently unveiled visual search interface - I think it augurs some serious new - and useful - approaches to sifting through massive amounts of related data.

From the Thomas Weisel's analyst coverage, sent to me in mail:

Google maintains dominance within "core search" but Bing Nudges Up m/m at Yahoo's and Google's Expense: Core search excludes searches conducted on video, local and map portions of the companies' websites. Google's U.S. query share of core search queries was down 11bps m/m to 64.6% in August but increased nearly 1.3 percentage points from August 2008. Yahoo's share was flat m/m at 19.3% in August and decreased 39bps y/y. Microsoft's share increased 35bps m/m to 9.3% in August and up 89bps y/y. Ask.com's share were was flat m/m at 3.9% in August but decreased 45bps y/y. AOL's share decreased 14bps m/m to 3.0% in August and decreased 133bps y/y.

Our take: Google continues to dominate audience market rankings in the U.S. while Microsoft has shown some signs of stabilization and a modest uptick with the launch of Bing in June. Yahoo, while having shown signs of stability over the past 12-18 months, has recently started to lose market share again, declining from 21.0% in January to 19.3% in August. Taken together, Yahoo and Microsoft represent 29% of the core search market in the U.S., flat with the previous month. Microsoft's new search engine, Bing, was launched at the beginning of June alongside an $80-100mn advertising campaign. This is the third month of data reflecting Bing's impact. While the data indicates a very modest near-term bounce, we will be watching closely to see if any query pickup is sustainable.

Alex Salkever has written a post on Facebok's self service CPC platform, which has been getting a lot of traction lately and is largely responsible for the company's recent boasting about being cash flow positive. From it:

I chatted with nearly two dozen people who are buying ads on Facebook. Many of them are also purchasing ads on Google (GOOG) and other online venues. The overwhelming sentiment? Facebook ads are actually more effective and do a better job of getting them in front of their target audiences.

The piece is worth reading and really contemplating. How many of you use Facebook ads? Do they work better than AdWords?

Danny Sullivan over at SEL has really teed off on Yahoo's search strategy, and any time he goes off, it's worth a read.

From it:

USER INTERFACE CHANGES WON’T LET YAHOO COMPETE IN SEARCH

Got it? Write it down, someone come check back on this in five years. If Yahoo’s moved up in search share thanks to outsourcing search and just toying with the user interface, I’ll eat those words somehow — covered even in Yahoo purple frosting.

No one has succeeded as a general search engine just by making user interface changes. No one, in the past nearly 15 years of us having search engines. That’s like 150 “real” years. (For more, see A Search Eulogy For Yahoo and Why Search Sucks & You Won’t Fix It The Way You Think)

The interview ticked me off in other ways. Bartz downplayed search as something people spend only 3% of their time on. Hey, I don’t spend all my time shopping. But who do you think makes more money off of me, places I shop at or television stations that deliver me entertainment?

Way to go dudes at Omgili!

By now you probably know about the “Search Options” feature Google introduced in May. One of its features is to limit the search results by time frame. By default the available time frames are: Any time, Past year, Past week, Recent results and Past 24 hours. Past 24 hours is nice but still far away from Real-time. What Google isn’t telling you is that you can search in the past minute and even in the past second. The trick is to change a parameter in the URL that will narrow down the time frames. ....Notice the URL parameter qdr:d. I assume qdr stands for Query Date Range (sounds about right). All you have to do to search for the query in the past minute is to change the parameter to qdr:n, and for the past second to qdr:s.

Past Minute:

http://www.google.com/search?q=barack%20obama&hl=en&output=search&tbs=qd

Bing is announcing new visual search features today. The post outlining it all is not yet up, but here are details and links from an email sent to me earlier:

Link to the blog post, not yet up, but soon they promise.

Link to the announcement on TC50 stage.

Text from the email, edited for clarity:

On Monday, Microsoft will launch a new beta feature in Bing, it's new decision engine, called Visual Search that is a new, easy way to search by clicking instead of typing. Visual Search helps you search information visually, and helps you refine a query when a picture makes it easier to sift through all the online information. Look for that movie you wanted to see, find the best new purse, or figure out which digital camera is right for you using an engaging visual experience without having to sort through page after page of links. People can try the beta of Visual Search by going to Bing.com/visualsearch.

· Visual Search categories that will be available on Monday are outlined below. This list will continue to grow and expand.

· Structured Content and images for Visual Search are provided by a number of sources, including MSN.

· The seamless transitions between selections are achieved through the integration of Silverlight technology

Visual Search Galleries:

Entertainment

100 heroes and villains, Billboard's past albums, Billboard's past songs, Film legends, Greatest movies, Movies in theaters

Popular books, Popular celebrities, Popular DVDs, Popular TV shows, Pulitzer winning fiction, Top albums, Top songs

Famous People

FBI's most wanted, Popular celebrities, US politicians, US presidents, US vice presidents, World leaders

Reference

Dog breeds, Periodic table, Travel destinations, US politicians, US presidents, US states, US vice presidents, World leaders, Yoga poses

Shopping

Cell phones, Digital cameras, Handbags, HDTVs, New cars, Popular books, Popular DVDs, Portable GPS, Pulitzer winning fiction, Top albums, Top iPhone apps

Sports

MLB players, MLB teams, NASCAR drivers, NBA players, NBA teams, NFL players, NFL teams, NHL players, NHL teams, UFC fighters

Update: I played with the Dog Breeds visual search, and found it pretty cool. It's not as deep as I would like - the promise is that you don't have to go out onto the web, and I found myself back into the "back and forth" button mode too soon, but the visual search is really cool to start with.

From Neiman:

Google is developing a micropayment platform that will be “available to both Google and non-Google properties within the next year,” according to a document the company submitted to the Newspaper Association of America. The system, an extension of Google Checkout, would be a new and unexpected option for the news industry as it considers how to charge for content online.

The revelation comes in an eight-page response to the NAA’s request for paid-content proposals, which it extended to several major technology companies and startups.

And from Google:

For us, search has always been our focus. And, starting today, you'll notice on our homepage and on our search results pages, our search box is growing in size. Although this is a very simple idea and an even simpler change, we're excited about it — because it symbolizes our focus on search and because it makes our clean, minimalist homepage even easier and more fun to use.

Well, if I were Facebook or eBay/Paypal, I'd be concerned about any payment system from Google, no matter how early stage. And the larger search bar, well, just seems to make sense. Search queries are getting longer, for one, and we're all getting older, for another - the text is now bigger as well. (OK, maybe it's just me getting older...)

My second post (of two) is up over at the BingTweets site, part of an FM partnership with Microsoft. In it I describe my frustration with search as it relates to helping me make a complicated decision: How to possibly buy a classic car. From it:

So first, how would I like to decide about my quest to buy a classic car? Well, ideally, I’d have a search application that could automate and process the tedious back and forth required to truly understand what the market looks like. After all, if I’m looking for classic Camaro or Porsche convertibles from the mid to late 1960s, there are only so many of them for sale, and they can be categorized by any number of important variables - price, model, region, color, features, etc. And while a number of sites do a fair job with a portion of the market, I don’t trust any of them to give me a general overview of what’s really out there. That’s where an intelligent search agent can really help.

But the next step is the harder one. I am not “smart” about how to buy a classic car. I don’t know enough to buy one with confidence. I don’t know what to ask about. I don’t know if it’s good or bad that an engine, electrical system, or transmission is original or rebuilt. I don’t know how one model does versus another in resale value, or insurance cost or…well, you get the picture. There’s a lot to consider, and I don’t know how to value everything. The world of classic cars is complex, like most major decisions. In short, there’s no easy way to decide in this case (unless, of course, I could just buy the most expensive one. That usually guarantees you’ve gotten what the market thinks you paid for it. Not an option for most of us).

So what do I need? I need help from a human being - someone I trust who has command of the classic car domain *and* has my best interests at heart. But given that I don’t have a spare Uncle who happens to be a classic car nut, what am I to do?

Yesterday I got a chance to debrief with two leaders of Yahoo's search team (yes, I know how that sounds given the Bing deal, but bear with me here). Late last week Yahoo announced its intentions with regard to continuing its innovation in search, and I had noted the irony of such an announcement.

I think most of the industry has written off Yahoo as a search player, and for some good reason. It's true the company has abandoned two key pieces of the search puzzle - indexing and search monetization. But it's also true, as I noted in my coverage of the deal, that Yahoo is retaining its right to control the user interface to search, and it's clear that's what the company is now focusing on.

What I find fascinating about this is how clearly it positions Yahoo to compete, directly, with its partner Microsoft and Bing. More on this later today.

It feels a bit odd to be writing that headline - "Yahoo's New Search" - given the company's deal with Bing/Microsoft. But Yahoo seems intent on declaring its independence with regard to search, even as it sells its asset and audience away to its newfound partner.

Yahoo does retain, in the deal, the right to innovate on top of Bing results, and I guess that's where this announcement is pointing - noting that Yahoo has been innovating in search UI and plans to continue to do so. I'm talking with the Yahoo folks next week and will have more on their plans then. But it strikes me as potentially conflicting to the deal for Yahoo to be innovating in UI on top of Bing, as one of Bing's strengths is its innovation in UI....

As part of BingTweets, an FM/Microsoft promotion blending the two services, I was asked to opine on the idea of how we use the web to make decisions. My first post has been up for a while but I managed to lose track of time and forgot to let you all know about it. I wrote a piece called "Decisions are Never Easy - So Far" - and have already written a followup piece, though that one is yet to be published. (And yes, I've asked them to make that picture smaller. Migod.)

From the first post:

If what you are looking for is a hotel room, a plane ticket, or something else in the “head end” of search results, plenty of sites aggregate tons of results for you. But as soon as you go a bit down the tail - like my example for classic cars - search becomes a pivot point for an ongoing and often taxing decision process. The opportunity, I think, is to figure out a way to support that process down the tail - saving us time, clicks, and frustration along the way. I see two paths toward that goal: one is creating applications on top of “ten blue links” which help me organize and aggregate the knowledge I process while pursuing a search query, and the second is making my searches social, so I can share the process of learning and learn from those who have shared - not unlike Vannevar Bush’s “Memex” concept.

When the second piece is up, I'll post an excerpt here as well.

Back when I predicted this in January, I recall worrying I was calling it too early. Now it appears the timing was about right. From Mashable:

...while Google grew from June to July, it still lost market share to its competitors – from 66.1% in June to 64.8% in July, a 1.3 percentage point drop.

From my prediction: 3. Google will see search share decline significantly for the first time ever. It will also struggle to find an answer to the question of how it diversifies its revenue in 2009.

There's more to be said on that second point, revenue diversification. More on that after the summer break I'm supposedly on.

Matt Cutts points to a video interview (embedded above) on Google's Caffeine infrastructure update.

"It's a pretty fundamentally big change" Matt says. What I'd like to know is why and in response to what changes on the web. Of course, the major changes in how the web works are clear: Real Time Search.

In this post (and/or this one) I said:

In short, Google represents a remarkable achievement: the ability to query the static web. But it remains to be seen if it can shift into a new phase: querying the realtime web.

It's inarguable that the web is shifting into a new time axis. Blogging was the first real indication of this, but blogging, while much faster than the traditional HTML-driven web, is, in the end, still the HTML-driven web.

Part and parcel to this shift is the web's adoption of Flash/Silverlight/Ajax - a shift to assuming the web works in real time, like an application on your desktop. That makes it damn hard to index stuff, because pages are not static, they are created in real time in response to user demand. This is a new framework for how the web works, and if Google doesn't respond to it, Google basically will become relegated to a card catalog archive of static HTML pages. No way will Google let that happen...

(By the way, one of the reasons I was impressed with Wowd was exactly because of its ability to, at scale, track a new signal in the web - the signal of what we are actually doing in real time...as opposed to the signal of the link...but more on that later.

Matt was asked if Caffeine was specifically about Real Time, and he was not totally specific about this but it's pretty obvious it is all about this shift.

Oh, and Matt says it's not because of Bing. In one way, I agree. But let's be real. Microsoft and Yahoo did this deal because Yahoo alone could never sustain the infrastructure costs associated with indexing and processing the Real Time Web. So in truth, Google did this because it had to, just like Microsoft and Yahoo did what they did because they have to. If you want to play, you have to get the infrastructure right.

Facebook's previously announced realtime engine has been released, coverage from Mashable:

Fast forward to today: Facebook just announced that it is rolling out the new Facebook search. With realtime search and FriendFeed in its pocket, Facebook is gunning directly for Twitter.

Also for Mashable, a story on Google's "major revision" of its engine. I plan to dig into this one, as I sense it has a lot to do with crossing the infrastructure chasm to real time:

Secretly, they’ve been working on a new project:the next generation of Google Search. This isn’t just some minor upgrade, but an entire new infrastructure for the world’s largest search engine. In other words: it’s a new version of Google.

The project’s still under construction, but Google’s now confident enough in the new version of its search engine that it has released the development version for public consumption.