free html hit counter Of Note in Search Biz Archives - Page 7 of 181 - John Battelle's Search Blog

The Database of Intentions Is Far Larger Than I Thought

By - March 05, 2010

Screen shot 2010-03-05 at 9.01.41 AM.pngWay back in November of 2003, when I was a much younger man and the world had yet to fall head over heels in love with Google, I wrote a post called The Database of Intentions. It was an attempt to explain a one-off reference in an earlier post – but not much earlier, as the “DBoI” post, as I call it, was just the sixty-third post of my then-early blogging career. (This is the 5,142nd, by comparison…)

I had, in fact, been ruminating on this concept for over a year, driven by an Holy Sh*t moment in late 2001 when Google introduced its first ever Zeitgeist round up of trending search terms. Scanning the lists of rising and declining terms, I realized that Google – not to mention every other search engine, ISP, and most likely every government – had in their grasp a datastream that, were they to just pay attention, could quite possibly be the most potent signal of human intentions in the history of the world.

Zeitgeist, it struck me, was proof that Google was indeed paying attention. I went on to write The Search, and Google went on to become, well, Google. My study of Google also led me to start Web 2, with Tim O’Reilly, and Federated Media, which I positioned as a media company that leveraged the impact of The Database of Intentions.

But over the past few years, as I’ve labored in the fields of digital media and marketing – mostly through my work at FM – I’ve come to revise my concept of what The Database of Intentions truly is. In my initial description, I limited the concept to web search and web search alone:

The Database of Intentions is simply this: The aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result.

At the time, that certainly seemed like a big enough idea. No such artifact had ever existed, and its implications were massive. In my 2003 post, I continued:

This information represents, in aggregate form, a place holder for the intentions of humankind – a massive database of desires, needs, wants, and likes that can be discovered, supoenaed, archived, tracked, and exploited to all sorts of ends. Such a beast has never before existed in the history of culture, but is almost guaranteed to grow exponentially from this day forward. This artifact can tell us extraordinary things about who we are and what we want as a culture. And it has the potential to be abused in equally extraordinary fashion.

Search was a pristine signal, an eruption of oxygen in the anoxic ocean of the early web, and an entire ecosystem grew in its bloom. The first implication was already manifest: Google had launched AdWords and AdSense, Overture (later to become Yahoo Search Marketing) was thriving, and a burgeoning paid search ecosystem was in the early stages of becoming a multi-billion commercial expression of the Database of Intention’s power.

But as anyone who’s been reading this site already knows, web search as a pure signal has been attenuating of late – overwhelmed by the sheer magnitude of data on the web, for one, and secondly by our own increasingly complicated expectations.

Nature abhors a vacuum, and so does the Internet. In the past year I’ve come to the conclusion that “web search” was just the first of many fields in the Database of Intentions. For those of you who are not database geeks, and to further pad the metaphor, a field in a database is colloquially defined as a specific type of information in that database. Sets of fields are called records, and sets of records make up the database.

My mistake in 2003 was to assume that the entire Database of Intentions was created through our interactions with traditional web search. I no longer believe this to be true. In the past five or so years, we’ve seen “eruptions” of entirely new fields, each of which, I believe, represent equally powerful signals – oxygen flows around which massive ecosystems are already developing. In fact, the interplay of all of these signals (plus future ones) represents no less than the sum of our economic and cultural potential.

By now you’ve probably already guessed what these new signals might be. I’ve made a rudimentary chart, but to narrate:

(NB: i’ve updated the chart here with a field for commerce…)

Fields in the DBoI 3.2010.png

The first signal, of course, was The Query. A query was a declaration of a very particular intent: What I Want from the web. Sure, it has many permutations – navigation, commerce, informational, etc. etc., but in essence, the goal was to find something you wanted. Hence the name search, after all.  

The next signal to emerge is The Social Graph. With this signal we’ve declared not only Who We Are, we’ve also declared Who We Know. Both are powerful intent-driven declarations, and both have deep interplays with search. By manifesting who we are and who we know, we can find and be found by others.

The third signal emerged almost simultaneously with The Social Graph – The Status Update. This is a personal declaration of what we deem important, noteworthy, shareable: What’s on our minds, what’s happening, what’s worthy. Again, a powerful search signal, in particular in real time.

The latest signal is The Check-in – or Where I Am. This is a crowning declaration of intent, in a fashion, because it connects the physical to the virtual, securing the Database of Intentions to the terra firma of the Real World. As with the other three fields, the check-in – which I expect will soon become automatic via our mobile devices – is a vastly powerful signal of intent: “I am here. So what you got for me?”

Taken together (and honestly, there’s really no other way to think about it, to my mind), these signals form a Database of Intentions that is magnitudes of order larger, more complex, and more powerful than my original concept back in 2003. And while the current players in each category are clear, what’s also clear is that the battle is on to control each of these critical signals. Google, if you include its Local services, already plays in all of them, and I expect Microsoft will as well. Facebook may never play in “The Query,” nor will Twitter, but expect both to play in The Check-in, and soon. The newcomers? Well, most of us expect them to be acquired. Then again, that’s what we thought of Google in 2000, and Facebook in 2005. Why should Foursquare in 2010 be any different?

All of this begs a new definition of Search. I’ve often said that Search should not be defined by web search, but rather, by what a search is in the abstract. To my mind, each tweet or status update is a search query of sorts, as is each check-in and even each connection in the social graph. A more catholic definition of search would allow for a reconciliation of all these fields in the Database of Intentions. Regardless, it’s ever more obvious that while “traditional search” is reaching a plateau of sorts, at least in regards to how we understand its potential, when you add the new signals of social, status update, and check-in, we’re still in the very early stages of a distinctly punctuated phase of the Internet’s evolution.

I’m on the lookout for new Signals. I’m quite certain we’re not nearly finished creating them.


NB: As a creator and publisher of media, one very strong conclusion can be drawn from all of this. If you’re not viewing your job to be a curator, clarifier, interpreter, and amplifier of the Database of Intentions, you’re soon going to be out of business. The Database of Intentions is the fuel that drives media platforms, and as I’ve argued elsewhere, every business is now a media business.

NBB: My thanks to the folks at Adobe and Omniture for the forcing mechanism of my keynote earlier this week, where I first organized the thinking above.

  • Content Marquee

Google Buys Human-driven Search Engine Aardvark: Will It Make It to the Main SERPS?

By - February 11, 2010

Screen shot 2010-02-11 at 11.44.03 AM.pngThe news broke today that Google will be buying Aardvark, a human (and algorithm) powered social search engine that I have written about quite a bit (early last year, most recently, all). I’ve also featured the service’s founders at both Web2 and the CM Summit.)

I’ve confirmed the news in an email with CEO Max Ventilla.

I can’t say I’m surprised by this news. Aardvark’s founders and advisers have strong ties with Google (Ventilla worked there, and a key adviser was at Kaltix, which was purchased by Google).

To me the critical question around this move is this: Will the Aardvark acquisition be a Dodgeball, or will it be a Applied Semantics? With Dodgeball, Google bought a promising startup in a strategically important space, but instead of integrating the technology and committing, it let it languish (the founders left and started Foursquare). Google later determined it must play in the space, and rolled out any number of features inside its mobile, map, and even Gmail products that mimic Dodgeball’s early features.

With Applied Semantics, Google again bought a promising startup in a strategically important space, but this time it successfully integrated the company’s technology and team, driving a crucial new business – AdSense – to become a critical and game changing business for the company.

So which is Aardvark? I’m not sure anyone at either company is sure, but Google is spending a reported $50 million to make sure no one else can find out. I do know Max well enough to say that his goal would be to see Aardvark integrated into the main search interface, such that when you ask Google a question, it would give you the option of “asking a human” through the ‘vark service.

Now that would be pretty cool.

Not to mention, ‘vark uses Facebook Connect as its core social graph for question answering. I certainly hope that will stand as the company integrates.

The parties can’t speak on the record about this yet, but another hope I have for this acquisition is that some of that ‘vark DNA about humans being critical to search – not as data points, but as part of the solution, connecting one to the other – will somehow infect the Google genome. We’ll see….

Updated: Google to Air "Search Stories" Ad During Super Bowl…

By - February 06, 2010

Remember when I wrote about the new “Search Stories” ads for Google’s core search offerings?

In that post, I noted “It’s truly a brand campaign: Google is not selling anything here other than its own brand – that ephemeral sensibility that resides between its customers’ ears.” Well I’ve got a pretty reliable source who is telling me Google plans to hit the branded advertising big leagues this Sunday – the source says Google’s “Parisian Love” ad (below) will air during the third quarter of the Super Bowl.

Now that would be a true turning point for the brand – a brand that, for nearly ten years, dismissed brand advertising as a waste of money (“The last bastion of unaccountable spending in corporate America,” in Eric Schmidt’s words back in 2006), and built its entire fortune on turning the advertising model upside down.

I emailed folks at Google for comment today, and a spokesperson said “Watch the Super Bowl!” That ain’t a no, folks. (It’s not a Screen shot 2010-02-06 at 3.29.23 PM.pngyes, either, but…)

I can’t find the ad in this lineup of SuperBowl advertisers, but I’d not be surprised if Google had asked CBS to keep their name out of the pre-game hype (my source was told Google was keeping this quiet). File this as a strong rumor for now, as I can’t get a secondary confirmation – though Google’s response was pretty telling.

Needless to say, I’ll be Tivo’ing the game….Here’s the ad.

UPDATE: After I emailed Google for comment, Eric Schmidt tweeted this out:

Can’t wait to watch the Superbowl tomorrow. Be sure to watch the ads in the 3rd quarter (someone said “Hell has indeed frozen over.”)

Eric, you trying to scoop my scoop?! Who ever would have thunk it?

Thursday Signal: Are You Checked In?

By - February 04, 2010

Screen shot 2010-02-04 at 11.08.18 AM.png Today is all about checking in. Not so much driven by anything in today’s news, but every week or so I’ll just go off based on what’s on my mind – driven by the news, to be sure, but also by the bricolage of a lot of inputs over time.   

And over the past few weeks, I’ve been developing a thesis around the concept of “checking in.” Now for those of you not playing along at home, “checking in” is the terminology for “declaring where I am and what I’m doing through mobile devices and social media platforms.”

As usual, I’m a late bloomer in this new trend. I joined Foursquare, one of several check-in-based services, about a month ago. I’ve started checking in at work, the gym, various restaurants and local businesses. The service has a strong game element, with social capital earned for checking in, or doing more than one thing in a day, or unlocking action-based “badges,” or repeat check ins over time (Foursquare makes you “Mayor” of a location if you check in there the most. Competition amongst Foursquare nerds is pretty intense for those Mayorships.)

Other services that employ checking in include GoWalla, Yelp, and MyTown. Twitter is adding location services as we speak, which is just another way of saying it’ll support checking in shortly (although most check in services drive announcement tweets already).

And while it may not be clear as to why, I fully expect Google and Facebook to follow suit by enabling some kind of check-in behavior shortly.

Here’s why. To my mind, checking-in is simply another use case on the evolutionary path of search. As I said in the book, each search query is a declaration of intent – you are telling that search engine what you want, and hoping the engine will return a result that satisfies that declared intention.

Checking-in is a powerful new field in the database of intentions. It is a social declaration that “I am here” and, in a more nuanced way, “I am open to appropriate responses/conversations based on the fact that I am here.” Whereas search intent is clearly a request for a specific response, check-in intent is less specific – and hence more open.

I expect this to evolve quickly. I can imagine a time, and it ain’t far off, when we set our mobile devices to automatically check-in at our favorite places, and expect that that check-in will reward us with localized and personalized offers, discounts, and social capital of some sort or another. Furthermore, I expect we will soon expect that if we set our device to “discovery” mode, local businesses (and random strangers too) will be able to ping us with enticements and announcements of all kinds.

Instrumentation of this new social/local/mobile reality will be initially clumsy and fraught, but not for long. The use case is simply too compelling. It’s already happening in various ways – the Chipotle burrito app, the Polo store. Imagine what happens when McDonald’s adopts it? Game changer.


In other news:

Is Amazon Building a Superkindle? (NYT) Yes, it bought a multitouch technology company, and yes, it’s going to get fun out there in ApplevsAndroidvsAdobevsAmazonLand.

Snickers Uses Social Media, SEM to Support ‘Lead Spot’ in Super Bowl Ads (ClickZ) More proof that social marketing is platform independent/supportive.

He Calls Google A Vampire, But Mark Cuban’s Mahalo Is Doing The Sucking (SEL) Oh SNAP.

Unclear ROI Impedes Mobile Marketing (MarketingProfs) You want proof of ROI? It’s coming. BTW, it’s also already here in terms of higher CTRs, if that’s your thing….(as anyone at AdMob or Microsoft Mobile Ads will tell you).

The IAgency: How the IPad Will Change the Advertising Business (AdAge) Or: Why We Should Emulate the Dying Publishing Industry. Yes please…do.   

Mobile Internet Market to Eclipse Desktop Internet (Brian Solis) Anyone who saw Mary Meeker at Web 2 last year already knew this but it’s worth repeating…

Foursquare Plots Its Business Model (BI) Tick, tick, tick….BOOOOOM.

SlideShare Launches Channels for Businesses and Brands (Mashable)

The Anatomy of a Large-Scale Social Search Engine

By - February 02, 2010

Screen shot 2010-02-02 at 6.02.56 PM.pngThe folks at Aardvark have posted an ambitious paper over on the ‘vark blog. Titled after Brin and Page’s original “Anatomy of a Large-Scale Hypertextual Web Search Engine”, the paper presents the Aardvark engine and, in its authors’ words: “describes the fundamental differences between the traditional “Library” paradigm of web search — in which answers are found in existing online content — and the new “Village” paradigm of social search — in which answers arise in conversation with the people in your network.”

I have read most of the paper, which has been accepted at WWW 2010 (it reminded me of all the search papers I read in preparation for writing The Search), and found a lot worthy of interest.

First, the paper’s authors, both of whom have worked at Google, clearly have a sense of potential history here, in that they not only crib Google’s original paper’s title, they also mirror the first line (substituting “Aardvark” for “Google”, of course). Now that’s some b*lls. Of course, when Larry and Sergey first presented Google, they couldn’t even get their paper accepted (it took three tries, if I recall correctly. Someone should write a book about that…).

Second, it’s unusual for a Valley startup to lay out its architecture and technological specs as willingly as Aardvark has. There’s a lot of math in here that I couldn’t parse even if I had the will to try.

Third, we learn some cool things about how Aardvark works. Check this quote out: “…unlike quality scores like PageRank [13], Aardvark’s quality score aims to measure intimacy rather than authority. And unlike the relevance scores in corpus-based search

Screen shot 2010-02-02 at 5.57.33 PM.png

engines, Aardvark’s relevance score aims to measure a user’s potential to answer a query, rather than a document’s existing capability to answer a query.”

Also interesting: ” this involves modeling a user as a content- generator, with probabilities indicating the likelihood she will likely respond to questions about given topics. Each topic in a user profile has an associated score, depending upon the confidence appropriate to the source of the topic. In addition, Aardvark learns over time which topics not to send a user questions about…”

There’s a lot more like this in the paper, it’s worth reading. The authors even did a test of Aardvark results against Google, with the results being something of a push (see the last page for details). Not bad for an upstart service.

Lastly, we learn a lot about the service, thanks to a number of charts, including something about Aardvark’s growth, which I had not really anticipated. It’s up and to the right, as you can see from the chart.

Google Rolling Out Social Search: But Does It Leverage Facebook?

By - January 27, 2010

Screen shot 2010-01-27 at 1.56.59 PM.png

Forget the iPad, today Google is taking another step toward its stated goal of “making search more social.” There’s a lot of goodness in here, in terms of features and approach, but it’s just silly to pretend you can do any of this without directly addressing the 400 million-person elephant in the room called Facebook. Put simply: I can’t figure out if this new service uses my Facebook social graph. And to my mind, that’s a problem.

From the blog post announcing the public beta of social search (first announced at Web 2 late last year):

We think there’s tremendous potential for social information to improve search, and we’re just beginning to scratch the surface. We’re leaving a “beta” label on social results because we know there’s a lot more we can do. If you want to get the most out of Social Search right away, get started by creating a Google profile, where you can add links to your other public online social services.

Indeed – a lot more, like make it really easy to use your Facebook social graph, the way tons of other sites and apps do. Why not just use Facebook Connect? Hang on a tick, the video giving us an overview of the service says once you create that Google Profile, you can add connections via Blogger, Twitter, and “any other online networks you might be a part of” (45 seconds in). Might that include Facebook?

OK dear readers, I’m going to do it. I’m gonna make a Google Profile, just to find out…. Well, I’m still a bit perplexed. You can add any URL as a “Link” in your profile, so I added my Facebook pages. However, once I got through the initial form (which was not simple – I had to fill out all the info I already did with Facebook and LinkedIn, and my own name is not available as a profile URL, not /johnbattelle, not jbattelle. Darn! I picked /johnlinwoodbattelle, so now you all know my middle name…) Er, anyway, there *was* a prompt to “Share It On Facebook” after all that…

Aha! Maybe this will get my Facebook social graph goodness into Google Social Search?

Not that I could tell. Just a simply “share on Facebook” implementation, declaring my profile to my FB pals. But no deep integration. As far as I can tell, my Facebook social graph will not inform my social searchin’ on Google. As I understand it from reading previous coverage of the product, Google social search *will* leverage FriendFeed, recently purchased by Facebook. But as far as I can tell, it does not leverage Facebook proper.

And that, to my mind, is just silly. Silly in the main, because as a consumer, clear, direct, and transparent integration with Facebook would be a huge *win* for my understanding of Google’s social searching. Wouldn’t it? Or am I missing something? (Besides the competitive issues, of course…)

I’ve pinged Google and other sources to find out if I’m just deeply in the dark….

Update: Google has provided me an answer to my initial question:

“If someone links to his Facebook account from his Google profile, Social Search may surface that user’s public profile page. These are the same public profile pages already available on a search of and other search engines today. While we’re interested to continue expanding the comprehensiveness of Social Search, we do not currently use your Facebook connections as part of Google Social Search.”

What I’d like to know then is this: Why not?

The Evolving Search Interface: Mobile Drives Search As App

By -

Screen shot 2010-01-15 at 11.10.13 AM.png

I’ve said before that search interfaces, stuck in the command line interface of DOS, will at some point evolve into applications on top of a commodity search index. I further opined that Bing, in particular Bing’s limited but compelling visual search, was just such an example: search as an interactive, rich application, as opposed to search as a list of results.  

The commodity of search results is critical, but as we shift our usage to the mobile web, the use case for a list of results weakens. Instead, as this Bizweek article points out, we’re using apps. On their face, these apps don’t seem like search at all. Except they are.

Take the popular iPhone app Exit Strategy, for example (at left). The app helps folks navigate the NY transit system. In essence, it consolidates a subset of search queries and answers them with a combination of domain-specific structured results and an elegant user interface. The structured dataset is the NY transit map and schedule, the UI is based on the iPhone’s unique ecosystem of interface. The result: No one with this app is Googling “best route Bronx Midtown“. Instead, there’s an app for that.

Google can’t help but see this as a threat. For nearly every structured set of results, there’ll be an app for that, if there isn’t already. To my mind, the question becomes one of using search to find the best apps. I wonder how Google is surfacing iPhone apps as answers to questions pertinent to destroying its own query volume? For it seems to me that a very good result for the query above, if done on Google over an iPhone, would be “Exit Strategy.”

Huh. Yet another reason to lean into Android, no doubt.

An Apple Search Engine?

By -

….driven by the need to kick Google off the iPhone? An interesting idea. Worth thinking about….

From a Businessweek article:

Some analysts believe the Apple-Google battle is likely to get much rougher in the months ahead. Ovum’s Yarmis thinks Apple may soon decide to dump Google as the default search engine on its devices, primarily to cut Google off from mobile data that could be used to improve its advertising and Android technology. Jobs might cut a deal with—gasp!—Microsoft to make Bing Apple’s engine of choice, or even launch its own search engine, Yarmis says. “I fully expect [Apple] to do something in search,” he adds. “If there’s all these advertising dollars to be won, why would it want Google on its iPhones?”

Oh, the Humanity: The Database of Intentions At Twitter Is Empty (After Two Weeks)

By - January 14, 2010

I was stunned to learn, via Danny, that our collective tweets seem lost to eternity (or at least, to search). While the data exists, tweets can’t be found via search, which means they can’t be found via the search API, which means…well, they can’t be found. I hope this situation is rectified, if only for history’s sake.

(Danny notes that they can be found using Google or Bing, at least for now. That’s a relief. But it does not bode well for Twitter’s ability to scale.)