John Battelle's Searchblog

It's taken a while, but I finally have time to rewrite the post I wrote this morning about Facebook search. For some reason my blog editor ate the post, something that has never happened to me and really threw me off.

In any case, this morning I noticed a post on Mashable about Facebook's new "superfresh" search plans - in essence, a plan to make the Facebook newsfeed searchable, and most impressively, to filter that through your social graph. In short, this is a Twitter search competitor with a Facebook twist, and while I think it's a fine move, it's nowhere near where Facebook needs to be in terms of search, and it seems a bit myopic: Facebook is way more than its newsfeed, and its search play is key to proving that value, and extending it.

First, a minor rant. Facebook search circa 2009 is akin to Alta Vista search circa 1994, or Ebay search circa 2004: very dumb and entirely lacking in structured, intelligently parsed data. In fact, it's worse that those two examples. It's clear that there are almost no intelligent signals in the way Facebook does its internal search, and I can't imagine anyone is happy with it. A few examples:

Here's a search for "john battelle status" as of today. There are no results. How on earth can that be? Not even a referral to my status updates? The engine clearly doesn't understand the concept of "status" which on Facebook, seems a crime.

Here's another one:

This is a search for "graffiti application". It does not find the popular application, Grafitti, which has more than 10 million installs and over 2 million active users. Whaaaa?

I could go on and on, but that's not the point. The point is, Facebook search could get a lot better. And I am *sure* the company is deep in planning on how to take its search to a new level - no small feat, given the size and scope of its service. No doubt building Facebook search today is akin to building Google ten years ago - bigger, most likely, in terms of data, algorithmic, and platform challenges.

So given the company is working on it, let's give them some input. What do we want it to be? Here are a few ideas I have, I'd love to hear yours:

- Leverage the social graph in search. When people search for other people (most likely the highest percentage use case on Facebook), show me that person's friends. Linked In does a very good job of search features like this, and is only getting better at it.

- Rethink how results are presented. Currently, it's all about pages on Facebook. Why? Why not think about search results in a similar manner to how we all understand search - multiple results, easily scanned, with short descriptors of what the link will bring us? There's a lot of room to innovate on top of this interface, but it's table stakes at least.

- Make search social. Show me what others are searching for, trending searches on the service, popular "found" items. Search is a signal, use it!

- Make search results linkable. When I do a search on Google, I can link to it. Here's a link to a Google search for "graffiti application," for example. And yes, the first result is the right one...

- Give me image search. I want to see pictures related to the results - Facebook is a highly visual service, so surface that!

These are off the top of my head (for the second time - I had others that I cannot recall...), but you get the picture. What do you want to see in Facebook search?

I have not yet grokked Hunch, the decision engine from flickr co-founder Caterina Fake and co., but from the coverage, the launch is a hit. Here's Caterina's post announcing it...

It's been a while since I've seen click fraud in the news, but this Times story caught my eye, in particular because it was Microsoft. Google usually gets all the headlines around this issue, but it's interesting to see Microsoft leading the charge in this arena. The story is worth reading, it sheds some light on the darker underpinnings of the search economy. From it:

Microsoft’s theory is that Mr. Lam was running or working for low-ranking sites that took potential client information for auto insurers. The complaint said that he directed traffic to competitors’ Web sites so they would pay for those clicks and exhaust their advertising budgets quickly, which let the lower-ranking sites that he sponsored move up in the paid-search results.

When people clicked through to his site, it asked them to supply contact information, which he then resold to auto insurance companies, according to Microsoft’s complaint, which estimated his profit at $250,000. In the complaint, it also said it had to credit back $1.5 million to advertisers because of the Lams’ alleged fake clicks. Microsoft is seeking $750,000 in damages from the defendants.

Although small advertisers have sued search firms, complaining the firms did not do enough to prevent fraudulent clicks, this is among the first cases where a search provider has gone after a suspected perpetrator.

Much buzz over the past few days about Facebook's plans to let folks (and, ahem, brands) claim their namespaces on Facebook. IE, Starting this weekend, I should be able to claim www.facebook.com/johnbattelle, just like I already "own" www.twitter.com/johnbattelle (sort of).

Anil Dash has a very funny send up of all this in a future forward timeline satire here. His point is - why is everyone falling all over themeselves to get their vanity URL on Facebook - or Twitter, or anywhere else for that matter - when the web is an open place and anyone can get their own URL, after all.

Well, yes and no. I've been complaining about Facebook's terrible link structure for a long time. We all spend time there, and create and share value there, but up till this weekend, it's been very difficult to point folks to places *inside* Facebook from places *outside* Facebook. The future of the web is ecosystemic - it's not about being in one place - this blog, that Twitter feed, or that Facebook page, it's about the ability to be anywhere, depending on the context and the moment. Sewing it all together is critical, and this move should make Facebook that much easier to incorporate into an ongoing, web wide conversation. I hope.

Just for the record, noting this article from the NYT on Google's continued skirmishes with the DOJ, this time on the book settlement front.

Quite the kerfuffle brewing over this site, built by a Microsoft employee, given folks the chance to blind taste test Bing, Google, and Yahoo. Bing was doing well early, but that might be due to the fact a lot of Microsoft folks took the test first. In any case, SAI has a good write up of the whole affair...  

My issue with this is that it's just about ten blue links. Bing in fact is about an application, including a good UI, on top of the base of ten blue links.

Steven Johnson has written for Time what I wish I had the time to write:

Skeptics might wonder just how much subversion and wit is conveyable via 140-character updates. But in recent months Twitter users have begun to find a route around that limitation by employing Twitter as a pointing device instead of a communications channel: sharing links to longer articles, discussions, posts, videos — anything that lives behind a URL. Websites that once saw their traffic dominated by Google search queries are seeing a growing number of new visitors coming from "passed links" at social networks like Twitter and Facebook. This is what the naysayers fail to understand: it's just as easy to use Twitter to spread the word about a brilliant 10,000-word New Yorker article as it is to spread the word about your Lucky Charms habit.

Put those three elements together — social networks, live searching and link-sharing — and you have a cocktail that poses what may amount to the most interesting alternative to Google's near monopoly in searching. At its heart, Google's system is built around the slow, anonymous accumulation of authority: pages rise to the top of Google's search results according to, in part, how many links point to them, which tends to favor older pages that have had time to build an audience. That's a fantastic solution for finding high-quality needles in the immense, spam-plagued haystack that is the contemporary Web. But it's not a particularly useful solution for finding out what people are saying right now, the in-the-moment conversation that industry pioneer John Battelle calls the "super fresh" Web.

Thanks for the call out, Steven, and the pointer, Fred, and the original idea, Yusuf (Medhi, at Microsoft, who really should get the credit for the term).

That's Carol Bartz, Yahoo CEO, who I swear I heard say the opposite (read about 2/3rds down) just a week ago at the D Conference. Perhaps she's finally putting the ghost of failed negotiations behind her, or, perhaps she's just furthering them. What do you all think?

Read this: Twitter = YouTube. Then read this: Visits to Twitter Search Soar. Discuss.

While the company still is mum on what it's announcing today at the D Conference (update it's now official....), everyone at the event presumes Steve Ballmer will be debuting Microsoft's new take on search, widely rumored to be called Bing. I am not certain there will be wifi coverage in the main ballroom, but it there is, I will be liveblogging his talk starting at 8.30 am PST. If there is not, look for my updates via SMS on Twitter.

Some of my notes:

Steve is being very Steve - emphatic, in selling mode, drinking a massive Starbucks iced coffee. Discussing why Bing, why now he says Microsoft decided search needed a new brand, but: "Building a brand is a step, step, STEP (almost yelling)!" Bing is a brand that consumers can get their mind around. We have to grow significantly relatively to where we are. We have to build brand equity as well as technology equity.

Walt Mossberg asks why "Bing"? Not Bing Crosby?! No, Steve says, it's short, it works globally, they could get the URLs, makes for a good brand.

On to the demo. Yusuf Mehdi joins Steve on the stage.

Live version goes up June 3.

He shows the Bing home page. Steve interrupts: "This is not something that pushes us to 80%" market share. But it's got some good stuff.

New features:

They point out that 25% of clicks on search is on the back button - IE, folks are not getting what they want. So they are focusing on fixing that.

"Best Match" - the one that is most authoritative. Does a search on Cannes film festival and the results are very good, including better answers instead of "following a million links".

The snippet is far more advanced.

A search for UPS shows just what Bing thinks most folks are looking for. Including customer service numbers. Live search for Microsoft shows their customer service number. That gets applause from the crowd.

The focus is on head end and service related answers like for flights, etc. It's going over well.

Another focus is helping with research and tasks.

"Taking a bet that search engines are not designed well to help with research."

The left nav bar is a big change. It uses a lot of key space to do navigation - essentially taking core search and laying an application interface over it. This is an important new approach.

I like this - the idea of seeing search as a core dataset upon which a more sophisticated navigation interface is overlaid. I've been talking about that for a while....

The big difference is that Bing brings most of the web woven into the search interface, instead of the way most of us search now, which is to go back and forth, back and forth.

Universal search and integration of images, videos, verticals is quite good.

Key verticals supported so far are travel, commerce, and I think health...though have to check on that.

Advertising as a business model on the internet is less proven that conventional wisdom, says Steve, talking about new models of making money with search, including doing deals with content providers to integrate into Bing. This raises the spectre of paid inclusion, worth pushing into on that front...

We're not trying to live off other people's work....says Steve, dig at Google.

Yusuf leaves stage, Steve sits down with Walt again. Walt asks will this really change things? Steve says his timeframe is lots of years.

Google is market leader. But Bing has "real differentiators" that will bring new folks to the engine. Walt asks about Ask spending money on marketing but then dropping off after showing some share gain. Steve says they plan to stay in the game long term (in terms of supporting brand with spending), and keep it up. Steve says it was a big budget. He had to gulp when he approved the budget. A gulp at a $60billion company is a big gulp.

Distribution is key (they are buying distro, more on that later - see my post predicting Microsoft would gain share back in january).

Steve called it "the little engine that could." Walt points out it's a pretty big train....

On my way to the D conference today, one of the main events is alleged to be the launch of "Bing," Microsoft's new search engine. I've been playing with it a bit, more on that later. Meanwhile, Microsoft plans a big marketing push, here's the news in Ad Age:

People with knowledge of the planned push said the ads won't go after Google, or Yahoo for that matter, by name. Instead, they'll focus on planting the idea that today's search engines don't work as well as consumers previously thought by asking them whether search (aka Google) really solves their problems. That, Microsoft is hoping, will give consumers a reason to consider switching search engines, which, of course, is one of Bing's biggest challenges.

TC notes the extraordinary growth Twitter has seen since its initial inflection. This is a growth pattern I have never seen in terms of speed - not in the nearly 25 years I've been watching this industry.

I think this is both Twitter's most important and dangerous phase of its young life. The retention problem must be addressed, and quickly. In my previous post about Twitter adding value to new users, I suggested Twitter incorporate some structure around its suggested users feature.

But with an inflection like this, I think it's time to swallow hard and embrace some serious social media jujitsu. In short, Twitter should integrate Facebook Connect in its signup process, and offer it as a feature for current users.

If it were to do that, then every new user who is also on Facebook (and who is not, at this point?) would be able to instantly follow Facebook friends who are on Twitter. Integrate this with a Groups feature, and you're proving value immediately.   

Talking with Whrrl CEO Jeff Holden yesterday, I mentioned this idea (I have also been bouncing it off a number of folks over the past month or so). He countered that such a move might add too much value to Facebook, but I disagree. Facebook has won the first round of the social graph game. Twitter has far more to gain by leveraging Facebook right now.

And after all, they can always turn the feature off it they want to.

What do you think?

Recall my piece on "from static to real time search," and read this coverage of Eric and Larry's take on Twitter:

Larry Page, Google’s co-founder, separately admitted Google needs to learn from Twitter and make its search engine operate at real-time speed.

Speaking at the Zeitgeist event during in a "fireside chat", hosted by Mr Schmidt, Mr Page admitted Google had so far “done a relatively poor job of creating things that work on a per second basis”.

“People really want to do stuff in real-time and they [Twitter] have done a great job about it... We will do a good job of things now we have these examples,” said Mr Page.

...and I have still not grokked the service. However, many have. Seems the reviews are mixed so far...

Gian Fulgoni, Executive Chair of Comscore, has an interesting analysis of what's happening in paid search lately. It's germane to my earlier posts about paid search share sliding and Google's decision to allow trademark ad bidding.

In his post, Gian notes that overall search queries are up dramatically (68% over two years) but:

if one looks at the number of paid clicks, the growth rate is a lower 18%, which raises the question: why have paid clicks grown 3x slower than the total number of queries?

Gian answers:

The reason appears to be that the ad coverage (i.e. the percent of search results pages with a paid ad) has dropped from 64% to 51% of searches.

Here's where it gets interesting. Why has ad coverage dropped? Gian has two hypotheses. First, search engines are getting better to reduce less relevant advertisers from the mix. But the second reason points to a more important potential breakdown in the AdWords model:

An analysis of comScore data shows that search queries are actually getting longer and that as searchers become more experienced they are using more words per search query. And this apparently reduces the likelihood that an advertiser has bid to have his/her ad included in the results page from these longer queries, due to paid search advertising strategies that limit ad coverage, such as Exact Match, Negative Match, and bid management software campaign optimization.

In short, our queries are getting closer to real conversation, real natural language, and Google's algorithms are having a harder time keeping up - matching advertiser demand to our increasingly complex queries.

As Gian said, fascinating.

Also worth noting, my pal Chas's analysis of what the decline in paid search means for brand advertising.

...and so will legal challenges, many of which are already underway.

Google's blog post is here.

Details: Google will now allow advertisers to bid on trademark terms, even if they don't own the trademark, so, for example, a local hardware store can bid on "Buy Makita Saws here" or Best Buy could bid on "Best Prices for Sony Plasmas".

Also, Google has opened up bidding on trademarked terms - so that competitors can bid on their rival's terms. It has been allowed in just four countries - US, UK, Canada, Ireland - but now will be allowed in 190.

Both moves mean more revenue for Google. Of course, the company says it's doing this "in an effort to improve ad quality and user experience." Which in fact, is true.

But...it also can be read as a sign that the company is doing what all large companies do: fine tuning its profit machine to yield more revenue.

One analyst, Ben Schacter at Broadpoint, put it this way in an email flash note sent out to press and clients:

The bottom line is that these two changes will be positive revenue drivers when allowed and into 3Q and beyond, however, we believe trademark holders will undoubtedly, and loudly, raise legal challenges.

According to HitWise, all is not well in the land of paid search:

Hitwise data indicate that the share of search traffic coming from paid listings is decreasing at the expense of organic traffic. In the four weeks to May 9, 2009, 7.25% of search engine traffic to All Categories of websites was from paid clicks. This compares to 9.84% in the same four week period in 2008 - representing a 26% decline in the share of paid clicks. This trend is apparent across 16 of the 17 Hitwise parent categories (i.e. Automotive, Food and Beverage, Health and Medical, etc). The only category that didn't see a decline in paid traffic was Education, which received 1.45% of search visits from paid clicks compared to 1.39% last year.

Here is where Google's 2004 IPO claim to not manage the company for quarterly earnings will be tested.

The Google Twitter Facebook goat rodeo is getting more interesting. At its Searchology event this week, (TC/Post coverage), Google unveiled a suite of new offerings that feel reactive to various competitors, including "Google Squared," a Google Labs response to Wolframs' new Alpha (more on that soon).

Reuters bills it this way:

Google also showed off a new feature, available immediately, that lets users view only the most timely search results, narrowing the results for a topic to the past 24 hours or the past week.

Google said it will search blogs and news sites, as well as the general information available online, to provide a fresher picture of certain subjects.

This feature comes amid the rising popularity of so-called real time search products, like the search feature on microblogging site Twitter, which allow users to search up-to-the-minute developments about certain topics online.

There is a lot to digest, will be doing it Friday to Monday, when I have time to write....

First two headines from IWantMedia today:

Here are the actual stories:

Google May Add Twitter-Like Features

Twitter to Expand Search Functionality

Yesterday Twitter rolled out integrated real time search to its entire user base, no small feat, given how fast that base has grown. It's pretty elegant, with Trending Topics searched for on the right, and onoing, constantly updated searches integrated into the same interface as normal Twitter.

This is a big deal for the company. I'd love to see how search volume grows. Also important is that the interface now mixes search queries and tweets interchangably, a key step toward the monetization platform I've called TweetSense. It's getting darn interesting, eh?