John Battelle's Searchblog

In the personality-driven world that is our industry, Qi Lu stands out for his relative lack of public profile. Widely respected as a technological leader while heading up search at Yahoo, Qi burst onto the industry scene when he defected to Microsoft last year and took the role of President of the Online Service division. In short, Qi is the man in charge of Microsoft's online strategy.

Our interview later this month will mark Qi's debut on the Web 2 stage. From all accounts, Qi is a very different character from his boss Steve Ballmer (who was a highlight of Web 2 two years ago). I'm looking forward to our interaction. Clearly we have a lot to discuss - the shifting sands of alliances (Facebook, Yahoo, Myspace, etc.), the rise (and fall?) of Bing, the Yahoo search deal, the future of MSN with regard to content, the role of ad exchanges and platforms (the Aquantive deal), and much more.

But I digress. What do *you* want to hear from Qi this year?

Others we'll be interviewing (and I've asked for your help):

Carol Bartz

Evan Williams

Brian Roberts

Jeff Immelt

To come: Aneesh Chopra, Sheryl Sandberg, Jon Miller, Austan Goolsbee, Paul Otellini, Shantanu Narayen, Tim Armstrong, Tim Berners Lee, and more. Again, an amazing lineup.

If you want to come, I can still get you a Searchblog discount (for about another week). Just ping me here.

Thanks to the BingTweets program, I've been asked to opine on search and decision engines. I'm kind of proud of my third and final post, which riffs on the first two and goes a bit, well, meta. I'd love to know what you guys think of it. I'll repost the first half here, and link back to the whole post on the original site that commissioned the work.  

Over the past two posts I’ve outlined my hopes and frustrations around search and decision making, using my desire to acquire a classic car as an example of both the opportunity and the limitations of web search as it stands today. As an astute commentator noted on my last post – “normally a 30 minute conversation is a whole lot better for any kind of complex question.”

Which leads me to my last post in this series. What is it about a conversation? Why can we, in 30 minutes or less, boil down what otherwise might be a multi-day quest into an answer that addresses nearly all our concerns? And what might that process teach us about what the Web lacks today and might bring us tomorrow?

Well the answer is at once simple and maddenly complex. Our ability to communicate using language is the result of millions of years of physical and cultural evolution, capped off by 15-25 years of personal childhood and early adult experience. But it comes so naturally, we forget how extraordinary this simple act really is.

I once asked Larry Page, co-founder of Google, what his dream search engine looked like. His answer: The computer from Star Trek – a omnipresent, all knowing machine with which you could converse. We’re a long way from that – and when we do get there, we’re bound to arrive a with a fair amount of trepidation – after all, every major summer blockbuster seems to burst with the narrative of machines that out think humans (Matrix, Terminator, Battlestar Galactica, 2001, I Robot…you get the picture).

But I have hope. Given this is my last post in the series, allow me to wax a bit philosophical. While we in the search and Internet industry focus almost exclusively on leveraging technology to get to better answers, perhaps we might take another approach. Perhaps instead of scaling machines to the point of where they can have a “human” conversation with us (a la Turing), perhaps instead (or, as well), we might leverage machines to help connect us to just the right human with whom we might have that conversation?

Continued...

Just off the phone with the folks behind Aardvark, a (relatively) new service that will most likely be the talk of SXSW this week. I'll have a longer riff on the company shortly, but suffice to say, I find it fascinating. The service rides between your social network(s), search, and the web, cleverly leveraging each to provide a platform for asking and answering the kinds of questions for which traditional search usually fails: the kinds of questions you ask a friend (or a friend of a friend, the real sweet spot here).

Many things make this company worth watching, its backers, its model, and its approach. Particularly noteworthy to Searchblog readers: a large group of the founders are from Google.

More soon.

Search, and Google in particular, was the first true language of the Web. But I've often called it a toddler's language - intentional, but not fully voiced. This past few weeks folks are noticing an important trend - the share of traffic referred to their sites is shifting. Facebook (and for some, like this site, Twitter) is becoming a primary source of traffic.

Why? Well, two big reasons. One, Facebook has metastasized to a size that rivals Google. And two, Facebook Connect has come into its own. People are sharing what they are reading, where they are going, and what they are doing, and the amplification of all that social intention is spreading across the web.

This is all part of the shift from static to real time search. Social is the fundamental element of that shift. What are YOU doing? What is on YOUR mind? Who do YOU want to SHARE it with?

Social search has been predicted (and funded) for years. It's finally happening. The conversation is evolving, from short bursts of declared intent inside a query bar, to ongoing, ambient declaration of social actions. Both will continue, but it's increasingly clear why Google's obsessed with Facebook (and Facebook with Twitter). And they are not alone.

One more piece falling into place in my five year old example of how search is changing via mobile. Snaptell. Found via the IPG Lab in LA earlier this week, though the company has been around for a few years. TC covers it back in November here.

From the wonderful Kevin Kelly:

While we have not yet made anything as complex as a human mind, we are trying to. The question is, what would be more complex than a human mind? What would we make if we could? What would such a thing do? In the story of technological evolution – or even biological evolution – what comes after minds?

The usual response to “what comes after a human mind” is better, faster, bigger minds. The same thing only more. That is probably true – we might be able to make or evolve bigger faster minds -- but as pictured they are still minds.

A more recent response, one that I have been championing, is that what comes after minds may be a biosphere of minds, an ecological network of many minds and many types of minds – sort of like rainforest of minds – that would have its own meta-level behavior and consequences. Just as a biological rainforest processes nutrients, energy, and diversity, this system of intelligences would process problems, memories, anticipations, data and knowledge. This rainforest of minds would contain all the human minds connected to it, as well as various artificial intelligences, as well as billions of semi-smart things linked up into a sprawling ecosystem of intelligences. Vegetable intelligences, insect intelligences, primate intelligences and human intelligences and maybe superhuman intelligences, all interacting in one seething network. As in any ecosystem, different agents have different capabilities and different roles. Some would cooperate, some would compete. The whole complex would be a dynamic beast, constantly in flux.

Make you think of anything?

I've decided to start linking to new stuff in search that gets sent to me again, but be forewarned - I won't be able to give it a full grokking. That said, a lot of new stuff has landed in my inbox, and it always bums me out to have to say "sorry guys I don't have time." So from now on, if something catches my eye, I'm going to link to it and give a (very) brief overview. If anyone out there wants to send me stuff, go right ahead, and know that whatever you mail to me I may borrow from to describe whatever it is you've built.

First up is Modista. It comes from Berkeley (GO BEARS) so perhaps that's what tipped it for me. From the email:

We are two computer science Ph.D. students at UC Berkeley, and we'd like to tell you about our project.
Modista applies visual search to online shopping, but it's very different from Like.com. We use the technology to enable product discovery, so users can browse huge inventories quickly and effectively. It fundamentally changes the user experience: rather than navigating text-based menus and scrolling through lists of results, you can simply rely on your visual intuition.

As we all know, our ability to sort massive amounts of information using our visual cortex far outstrips our ability to sort using textual analysis. I really like this tool, but the real test is whether my wife will!

Because, at the end of the day, Twitter shows the shift to the realtime web (Microsoft calls it "Live Search" and T'Rati called it the live web). And if Google doesn't own it, someone else will. More when, well, the holidays come, and I can write. Meantime, read the piece I referenced in the last post or the last tweet.

My post on the subject, while arguably arguable (yes, I know, I know, but it's better to just say it than let it stick in your craw) is up on the Looksmart Thought Leadership site (part of an FM program I am participating in). From it (this is just a portion):

I think Search is about to undergo an important evolution. It remains to be seen if this is punctuated equilibrium or a slow, constant process (it sort of feels like both), but the end result strikes me as extremely important: Very soon, we will be able to ask Search a very basic and extraordinarily important question that I can best summarize as this: What are people saying about (my query) right now?

When it first hit critical mass, it seemed Google answered this question. For the first time, you could ask a question in your native tongue, and get an answer. It felt immediate, but save for the speed with which the search results were rendered, it was not. Instead, it was archival - Google was the ultimate interface for stuff that had already been said - a while ago. When you queried Google, you got the popular wisdom - but only after it was uttered, edited into HTML format, published on the web, and then crawled and stored by Google's technology. True, that has sped up - Google indexes a lot of sites more than once a day now - but as it nears the event horizon, this approach to search won't scale.

In short, Google represents a remarkable achievement: the ability to query the static web. But it remains to be seen if it can shift into a new phase: querying the realtime web.

It's inarguable that the web is shifting into a new time axis. Blogging was the first real indication of this, but blogging, while much faster than the traditional HTML-driven web, is, in the end, still the HTML-driven web. To its credit, Technorati saw blogging as the vanguard of a shift to real time, and tried to become the first search engine for "the live web". It failed to gain critical mass, but I think the main reason was that the web was not yet "alive".

That is changing, rapidly. Yes, I'm thinking about Twitter, of course, which is quickly gaining critical mass as a conversation hub answering the question "what are you doing?" But I'm also thinking about ambient data more broadly, in particular as described by John Markoff's article (posted here). All of us are creating fountains of ambient data, from our phones, our web surfing, our offline purchasing, our interactions with tollbooths, you name it. Combine that ambient data (the imprint we leave on the digital world from our actions) with declarative data (what we proactively say we are doing right now) and you've got a major, delicious, wonderful, massive search problem, er, opportunity.

And with that search challenge comes an equally exciting monetization opportunity.

So here we go - Google is jumping into the social media search world. "SearchWiki" is Google's answer to the question "Why can't I make search work the way I want it to work, and share/learn from others doing the same thing?"

But one wonders if Google searchers have that question to begin with. As I've argued elsewhere, Google search had become a bit like the morning newspaper of yore - social glue that all of us could depend on because the results were pretty consistent. I don't believe that search shouldn't change - I'm a major proponent of change, particularly in the interface. But as Mike points out, many folks may not want this kind of change.

From Google's announcement:

Have you ever wanted to mark up Google search results? Maybe you're an avid hiker and the trail map site you always go to is in the 4th or 5th position and you want to move it to the top. Or perhaps it's not there at all and you'd like to add it. Or maybe you'd like to add some notes about what you found on that site and why you thought it was useful. Starting today you can do all this and tailor Google search results to best meet your needs.,,,Today we're launching SearchWiki, a way for you to customize search by re-ranking, deleting, adding, and commenting on search results.

..The changes you make only affect your own searches. But SearchWiki also is a great way to share your insights with other searchers. You can see how the community has collectively edited the search results by clicking on the "See all notes for this SearchWiki" link.

Clearly Google will learn a ton about search behavior through this new set of features, and presumably that will improve core search results. But what I find interesting in all this is what is says about what Google knows, and therefore decided to do. Google knows folks come to the site for repeat navigation - to find places they have already visited. And they know that they go there for discovery - to find things they've never visited but hope to find. A move like this seems to point Google toward bringing the two together, and potentially, re-portalizing the web.

What do I mean by that? Well, it's clear that Google is the starting point for a very large percentage of folks on the web. But while many of us start there, we don't spend much time there - we use Google as a way to jump from place to place. If, however, we can customize Google to become a one stop shop with all our favorite places, as well as comments and social connections, we may well change our behaviors and spend more time at that start place. While Google has never announced numbers, it's commonly assumed that its iGoogle start page is getting less than stellar traction. But iGoogle + SearchWiki? That just might do it. I'd like to post more thoughts on this development but the SearchWiki code has not propagated to my account, so I can't really test drive it. I'm sure when it does, more thoughts will come up....

Readers of this site will recall my ongoing insistence that voice will be the new search interface (and honestly, the next interface for much of the web). Earlier this week, a step toward that reality was taken by Google. It's going over well. From Cnet:

The new voice-activated Google Mobile app for the iPhone is finally here. Whatever the reason for the delay, it was worth the wait. As we wrote last week, the search app knows when you bring the phone to your face to speak into it. It beeps, you talk, and it executes a Google search on what you said.

Previous coverage of voice search on Searchblog.

From a Weisel report emailed to me just now:

On Monday (11/3) after the close, The Wall Street Journal reported that Google and Yahoo have submitted to the Department of Justice a revised version of their proposed search agreement. While we see little legal reasoning behind blocking the deal, we believe the DOJ is basically saying that Yahoo can't be trusted to do the right thing for its business over the long term.

Shortened Duration: The reported revised plan shortens the partnership from 10 years to 2 years, forcing Yahoo to avoid lowering its search monetization capability if the company can't rely on Google for a decade.

Cap on Outsourced Revenue: The revised deal would also place caps on the revenue that Yahoo can generate from the partnership to 25% of Yahoo's search revenues (or around $1bn annually based on our 2008 estimates). Given Yahoo had originally identified search revenues of $800mn that would be addressable for Google suggests again that the DOJ would want to put fail safe measures in place to limit Yahoo from getting too aggressive and outsource beyond the tail keywords which it had previously highlighted.

Here is a Reuters piece:

Yahoo Inc and Google Inc have drastically scaled back the scope of their search advertising deal, a person close to the discussions said on Monday, in a last-ditch effort to win U.S. antitrust approval.

The move comes after Google appeared to be on the verge of walking away from the partnership, which was announced in June to foil Microsoft Corp's takeover attempt of Yahoo. The deal has since drawn scrutiny from U.S. regulators amid a growing chorus of criticism from advertisers.

The two Internet companies have submitted a reworked proposal to the U.S. Department of Justice that shortens their partnership to just two years from 10 years, the source said.

Jerry and I sit down to talk on stage Weds.

I think the business model at Twitter is going to be really, really interesting, and I think it's going to leverage search, but search as a proxy for data and pattern recognition. We get an inkling of it at Election 2008, Twitter's mashup of Tweets relating to the election, but there's a lot more to think through. First off, Twitter is using its real estate to promote its deal with Current, which is a first, from what I can tell. The "ads" are on the right, right below each users' profile. I remember covering every new pixel as the Google homepage caved to promotional reality, it's interesting to watch it happen at Twitter, too, which I think has a lot of similarities to Google in terms of potential models.

Also worth watching is the hash function, where you can tag any topic (IE #redsox, as Churbuck pointed out). This function is not likely to catch on with my mother (I can't imagine her adding hashes to her tweets, much less tweeting...yet), but what it enables certainly could. The problem is, when you create a site to pull hashed stuff out into a stream the result is often less than useful (as Churbuck noted in his post).

This is where the role of curation and editors is paramount. Voice, as Fred pointed out. There is voice in editing, voice in curation. And voice adds value. And where value is added, marketers can play, both on Twitter (imagine a cars.twitter.com, with auto advertisers on the right rail and at the top, perhaps using contextual TweetSense - yes, it's owned, by...), and off (think about a feed of contextual Tweets and TweetSense next to conversational sites like Digg and, well, millions of others, as well as sites created simply from Twitter feeds on popular hashes...).

Just a (half) thought....

PS - why isn't search.twitter.com, where you can see hash streams, even promoted on the home page of Twitter? Am I missing something, as I usually do?

Not content to lease data from others who have satellites, Google today launched its own satellite into space. Via BeetTv, thanks Andy.

Talk about web meets world....this is yet another indicator of the integration of virtual and physical. And it brings Google one step closer to what I think could be the company's Waterloo - a viral meme that Google is sensing too much, knows too much, and is too powerful. It may not be rational, but no one ever accused humans of being entirely rational.

Update: Apparently Google does not own the satellite, just the data....

An Anthropology of Google's Search Experiments (with all data exposed, of course).

Never will happen, but we get some tantalizing hints in this post on the Google blog:

At any given time, we run anywhere from 50 to 200 experiments on Google sites all over the world. I'll start by describing experimental changes so small that you can barely tell the difference after staring at the page, and end with a couple of much more visually obvious experiments that we have run. There are a lot of people dedicated to detecting everything Google changes - and occasionally, things imagined that we did not do! - and they do latch on to a lot of our more prominent experiments. But the experiments with smaller changes are almost never noticed.

The ruling yesterday on the merits of Viacom's data requests is worthy of review. Ars has more here. I am preparing for a vacation and can't elaborate, but trust me on this one...

I am thinking hard about the impact of open search - the idea that a major search index becomes totally open to developers, an open API, etc. that allows search to become a true platform that people can develop on top of.

I'd love your thoughts on this....writing this soon....I'll update here with more thoughts but wanted to leave this as bread on the waters for the early risers...I know, I know, spam, but that can be routed around with business models and contracts...I've been noodling this for a long time and am close to saying SOMETHING....more background here (on Yahoo's search monkey) and here (when Amazon did it and no one seemed to notice...)...

This is a Big Deal. Now, I want to know: how will Flash files be ranked? Any ideas? Adobe is a major competitor to Microsoft in this front. How will Microsoft make Silverlight searchable? And will Google index all both equally? (My take: Oh yes it will. If it does not, that spells trouble in any congressional hearing...)

I will admit, I was entirely biased upon reading this story from Nick Carr, who has a knack for writing pieces that get a lot of attention by baiting his hook with contrarian link chum. Heck, he's really good at it, and I have a lot of respect for Nick. So I'll take the bait.

His piece starts by conjuring HAL, the famous AI which manipulates humans, then makes his case by citing his own "feeling" that Google has changed his attention span to somehow prove that search and web browsing in general is making us stupid.

Balderdash. What Carr is really saying is this: People are not reading long narrative anymore, and that makes me and my pals sad. So let's blame the Internet!

Sounds an awful lot like the complaints we heard about TV making us stupid. Did TV make us stupid? I dunno, ask Steven Johnson. I bet he has an opinion on this piece as well.

Carr writes: "Yet, for all that’s been written about the Net, there’s been little consideration of how, exactly, it’s reprogramming us. The Net’s intellectual ethic remains obscure."

So because Nick hasn't come up with a singular thesis as to what the "Net's intellectual ethic" is, we must declare it's making us stupid, eh?

Huh. He goes on to claim that Google is, in essence, an industrial style factory driven by a philosophy that is mechanizing our collective intellect much like factory automation mechanized our collective workforce - in short, Google is turn our minds into nothing more than collective cogs in some borg like hive mind. We're fucked, and it's all Google's fault.

Puuuuuuuhhhhleezzze.

Here's another quote: "The last thing these companies want is to encourage leisurely reading or slow, concentrated thought. It’s in their economic interest to drive us to distraction."

Right. And that's why Google encourages its workers to spend 20% of their time on passion projects. OK.

His conclusion: "As we come to rely on computers to mediate our understanding of the world, it is our own intelligence that flattens into artificial intelligence."

Good lord. Somehow Carr seems to presume that there's simply nothing valuable occurring in our minds when we engage with the extraordinary new medium of the web. Because we're starting to think in different ways, it must be bad. Right? Carr may believe that search and the Internet make us stupid, but I will counter his personal, anecdote-driven conclusions with one of my own: when I am deep in search for knowledge on the web, jumping from link to link, reading deeply in one moment, skimming hundreds of links the next, when I am pulling back to formulate and reformulate queries and devouring new connections as quickly as Google and the Web can serve them up, when I am performing bricolage in real time over the course of hours, I am "feeling" my brain light up, I and "feeling" like I'm getting smarter. A lot smarter, and in a way that only a human can be smarter.

And I have a feeling I'm not alone. What do you guys think?

Danny and I have contracted with Thomson Reuters, a sponsor of Searchblog, to write a series of posts on the future of search. They've given us no guidance, just asked us to ponder the topic. This is my first post, "A search is not just a search," longtime readers will find it familiar, if updated. From it:

In the past few years, a significant new feature has crept into the results portion of this otherwise predictable interface. Called “universal search,” the idea is to incorporate more than simple HTML pages into the results. A search for “London restaurants”, for example, might bring up maps and local results, as well as videos, images, organized reviews, and of course web pages. Every major search engine, from Google to Ask, has incorporated some kind of universality into its search results.

But while universal search points the way toward a new approach to getting you the answers you seek, it’s a half step at best. The results change, somewhat, but the process is pretty much the same. You enter a query, you get a set of results. Not particularly new.

What I find interesting are entirely new approaches to the interface of search.

We'll be writing one post every week or so for the next six weeks. I hope by the end it'll be an interesting body of work, it certainly will be if you give me input on what to think about, and critiques of what I've written. Thanks!