Way back in November of 2003, when I was a much younger man and the world had yet to fall head over heels in love with Google, I wrote a post called The Database of Intentions. It was an attempt to explain a one-off reference in an earlier post – but not much earlier, as the “DBoI” post, as I call it, was just the sixty-third post of my then-early blogging career. (This is the 5,142nd, by comparison…)
I had, in fact, been ruminating on this concept for over a year, driven by an Holy Sh*t moment in late 2001 when Google introduced its first ever Zeitgeist round up of trending search terms. Scanning the lists of rising and declining terms, I realized that Google – not to mention every other search engine, ISP, and most likely every government – had in their grasp a datastream that, were they to just pay attention, could quite possibly be the most potent signal of human intentions in the history of the world.
Zeitgeist, it struck me, was proof that Google was indeed paying attention. I went on to write The Search, and Google went on to become, well, Google. My study of Google also led me to start Web 2, with Tim O’Reilly, and Federated Media, which I positioned as a media company that leveraged the impact of The Database of Intentions.
But over the past few years, as I’ve labored in the fields of digital media and marketing – mostly through my work at FM – I’ve come to revise my concept of what The Database of Intentions truly is. In my initial description, I limited the concept to web search and web search alone:
The Database of Intentions is simply this: The aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result.
At the time, that certainly seemed like a big enough idea. No such artifact had ever existed, and its implications were massive. In my 2003 post, I continued:
This information represents, in aggregate form, a place holder for the intentions of humankind – a massive database of desires, needs, wants, and likes that can be discovered, supoenaed, archived, tracked, and exploited to all sorts of ends. Such a beast has never before existed in the history of culture, but is almost guaranteed to grow exponentially from this day forward. This artifact can tell us extraordinary things about who we are and what we want as a culture. And it has the potential to be abused in equally extraordinary fashion.
Search was a pristine signal, an eruption of oxygen in the anoxic ocean of the early web, and an entire ecosystem grew in its bloom. The first implication was already manifest: Google had launched AdWords and AdSense, Overture (later to become Yahoo Search Marketing) was thriving, and a burgeoning paid search ecosystem was in the early stages of becoming a multi-billion commercial expression of the Database of Intention’s power.
But as anyone who’s been reading this site already knows, web search as a pure signal has been attenuating of late – overwhelmed by the sheer magnitude of data on the web, for one, and secondly by our own increasingly complicated expectations.
Nature abhors a vacuum, and so does the Internet. In the past year I’ve come to the conclusion that “web search” was just the first of many fields in the Database of Intentions. For those of you who are not database geeks, and to further pad the metaphor, a field in a database is colloquially defined as a specific type of information in that database. Sets of fields are called records, and sets of records make up the database.
My mistake in 2003 was to assume that the entire Database of Intentions was created through our interactions with traditional web search. I no longer believe this to be true. In the past five or so years, we’ve seen “eruptions” of entirely new fields, each of which, I believe, represent equally powerful signals – oxygen flows around which massive ecosystems are already developing. In fact, the interplay of all of these signals (plus future ones) represents no less than the sum of our economic and cultural potential.
By now you’ve probably already guessed what these new signals might be. I’ve made a rudimentary chart, but to narrate:
(NB: i’ve updated the chart here with a field for commerce…)
The first signal, of course, was The Query. A query was a declaration of a very particular intent: What I Want from the web. Sure, it has many permutations – navigation, commerce, informational, etc. etc., but in essence, the goal was to find something you wanted. Hence the name search, after all.
The next signal to emerge is The Social Graph. With this signal we’ve declared not only Who We Are, we’ve also declared Who We Know. Both are powerful intent-driven declarations, and both have deep interplays with search. By manifesting who we are and who we know, we can find and be found by others.
The third signal emerged almost simultaneously with The Social Graph – The Status Update. This is a personal declaration of what we deem important, noteworthy, shareable: What’s on our minds, what’s happening, what’s worthy. Again, a powerful search signal, in particular in real time.
The latest signal is The Check-in – or Where I Am. This is a crowning declaration of intent, in a fashion, because it connects the physical to the virtual, securing the Database of Intentions to the terra firma of the Real World. As with the other three fields, the check-in – which I expect will soon become automatic via our mobile devices – is a vastly powerful signal of intent: “I am here. So what you got for me?”
Taken together (and honestly, there’s really no other way to think about it, to my mind), these signals form a Database of Intentions that is magnitudes of order larger, more complex, and more powerful than my original concept back in 2003. And while the current players in each category are clear, what’s also clear is that the battle is on to control each of these critical signals. Google, if you include its Local services, already plays in all of them, and I expect Microsoft will as well. Facebook may never play in “The Query,” nor will Twitter, but expect both to play in The Check-in, and soon. The newcomers? Well, most of us expect them to be acquired. Then again, that’s what we thought of Google in 2000, and Facebook in 2005. Why should Foursquare in 2010 be any different?
All of this begs a new definition of Search. I’ve often said that Search should not be defined by web search, but rather, by what a search is in the abstract. To my mind, each tweet or status update is a search query of sorts, as is each check-in and even each connection in the social graph. A more catholic definition of search would allow for a reconciliation of all these fields in the Database of Intentions. Regardless, it’s ever more obvious that while “traditional search” is reaching a plateau of sorts, at least in regards to how we understand its potential, when you add the new signals of social, status update, and check-in, we’re still in the very early stages of a distinctly punctuated phase of the Internet’s evolution.
I’m on the lookout for new Signals. I’m quite certain we’re not nearly finished creating them.
——-
NB: As a creator and publisher of media, one very strong conclusion can be drawn from all of this. If you’re not viewing your job to be a curator, clarifier, interpreter, and amplifier of the Database of Intentions, you’re soon going to be out of business. The Database of Intentions is the fuel that drives media platforms, and as I’ve argued elsewhere, every business is now a media business.
NBB: My thanks to the folks at Adobe and Omniture for the forcing mechanism of my keynote earlier this week, where I first organized the thinking above.
John, I have to be blunt.
Your notion of search is very shallow — it only touches the tip of the iceberg… at best.
You need to broaden you horizon to think beyond Google.
As a suggestion, please try to get in touch with Brenda Dervin. She has a quite holistic approach and might be able to provide a better overview of the vast field of information science better than I can. This field spans many decades of research, and in contrast you depiction of what you consider to be “search” is only a blip of the most recent developments.
I think if you take a step back and consider the broader picture, then you will begin to realize how amusingly limited your previous view was… indeed: it’s almost cute! (except that we’re also talking about a significant portion of the the success / failure of the global economy)
Such a gross mismatch cries out for correction, and therefore I think you would do well to broaden your scope.
🙂 nmw
John – My take is that status updates, checking in, or social graphs are really just refinements on intent. Not the intent itself.
An intent gets expressed as navigation, information or transaction. Intent can be refined by locality, context, relationships, or time. Maybe that’s what you meant in that the database has multiple attributes?
Anyway, good post. Made me think.
John, great post. You forgot a big one. Commercial intent, ie the things people buy. That data, formally locked up tight, is finally seeping out in a huge way.
Keep up the good work. It is great to see you writing again.
Great article and lots of food for thought. However, I would add that Facebook, MySpace and others either already are or soon will be playing “The Query” game.
Case in point – If you search something on Facebook, “Web Results” are shown and available.
I also feel that The Query is much more detailed than suggested here, and could be broken out into Relative Queries… based on location, real-time vs static, search vs research and result type, such as image vs document vs webspace vs Friends.
Completely agree John, and here’s another one to add: Driving Directions. They not only convey the user’s location, but where they’re intending on going (the same is true for all Travel-related queries).
Thanks for posting about this, I would love to read more about this topic.
Great post John. Wondering if there might be another “signal” to add to your database of intentions that has to do with some of the ways that your signals might be aggregated, filtered and most importantly presented -to facilitate discovery via those to whom you have granted greatest influence in particular/specific topic areas. And that is “What are people who I trust sharing and recommending across media” I am wondering if the recently announced ShareThis Stream (most shared topic streams) and video startup Redux (aggregating video via your social graph). ShareThis Stream here: http://sharethis.com/stream and Redux interview here: http://www.building43.com/videos/2010/03/02/redux-the-hottest-video-from-your-social-graph/
And if you include the recently launched PlanCast (http://www.plancast.com) you now have the “what am I planning to do” component.
John,
Very informative post, this clearly shows the evolution of the new web. What are your thoughts on merging the new web? Similar to the comment from Liz above, how do you see services such as Topsy, LeapFish , redux fitting in, sites that provide relevancy to said database of intentions by aggregating and filtering based on a specific query. Do you believe that as new information providers become relevant to the user such as the introduction of “check-ins”, tools will need to be readily available to harness said information?
Lena
another signal: what I play
collected currently by:
1) Gracenote (CDDB look-ups)
2) Last.fm (scrobbling)
3) Apple (itunes playcounts + genius)
4) MusicBrains (similar to gracenote)
John,
I enjoyed this, thought provoking. One signal I have been thinking about quite a bit is “How I feel.” My company has spent a fair bit of time trying to understand sentiment in the language of content (we use NLP algorithms to measure the subject and object of an emotive verb like “loved” or “attacked”, for example. Just pondering, but measured in aggregate over time, it would be interesting to see how expressed sentiment manifests itself as both a signal of intentions and a result of actual action against intentions. Anyway, thank you for the post.
-Will
the previous one’s field is probably called “The Playlist”
another field could be “the Trophies”
“What I’ve won” – badges, trophies and high scores
Current Players: Microsoft via X-Box Live, Sony via the PlayStation Network, Foursquare with their badges and any video game you ever played with a place to enter your high score.
Signals:
We’re all one family
We want to know and be greater (even in a selfless sense) – “subliminally” to be and experience the global family all at once
Cooperation & Tolerance = Peace
ultimately: http://www.da-peace.org/
Last one, I promise
Signal: “How I’m performing”
personal informatics from devices
Players:
1) Nike + (running data)
2) Weightwatchers (calorie intake)
3) LifeScan (blood glucose monitor for diabetics)
John,
Not to add to the noise (but of course I will) only material thing I think you’ve missed is “What I know”.
Ro
John, excellent thoughts here – I’m a HUGE fan of your book, which I read earlier this year.
While reading through it though I couldn’t help but relate most of your insights on the future of search, digital business models, and media more in synch w/ social networking than to traditional search. The database of intentions of today for me really is Facebook, not Google.
The internet is becoming less and less of a “world wide web”. Instead its structure is becoming more quantum-like. The future of search, I think, will push its results out to us, based on our social graph and digital footprints – the semantic web may very well be realized within Facebook.
Thanks for the comments guys. I think the one that really stands out for me is the commerce signal. It’s not really been a “public” signal – IE, broadcasted – and the data has been preserved by the commerce giants, not shared with consumers. I think that is going to change.
Well I agree with you 100% on that one (cf. http://snurl.com/link-advertising-free + I think you might recognize one of the examples I used 😉
Still, I suggest you check out the vast amount of research that’s been done for many decades already (and that some in Silicon Valley are only now discovering [or haphazardly re-inventing without the vaguest notion of a blueprint] … e.g. twitter getting into the field of “vocabulary control” — an area that’s indeed quite old, but has been overlooked by a widespread fanatical devotion to “just plain” bits).
🙂 nmw
John, a few years back I read your book, The Search, and I loved it, so thanks for writing that. At that time I was toying with the idea of becoming an entrepreneur. I did not jump into it completely but decided to invest some money in and help two friends with their venture. When we sealed the deal, I gave them two copies of The Search. I was inspired by the database of intentions but I was also inspired by what you wrote about search becoming more specific, i.e. developing alongside verticals. Healthcare search portals, food search portals, travel (which is what I am doing now with my own venture after my investment in my friends’ business didn’t turn out well). I believed then and I still believe that there’s a big opportunity for platforms that address certain areas of our lives rather than generalistic search as for google. This is for many reasons but also because a sector approach makes easier and better to tap into the local knowledge and the benefits of UG content.
I agree with your article above but I feel it’s incomplete as it doesn’t touch on the sector approach.
I believe the players you outline are there to stay and to change our lives more than it’s already happened (I have some reservations about Gowalla and Foursqaure becasue I don’t get excited by them, but it’s early to say). I think however there will be new players coming to the status and check-in stages; in part becasue I don’t think the current players have nailed it yet and in part becasue there’s a key element missing: the link between those ‘intentions’ and the ‘actions’ deriving from them, i.e. the actual purchase/consumption. I think Bing and Facebook are the closest of the lot to this and you need to add Amazon to the group.
I recall when I first read your book about the database of intensions and remember when the penny dropped for me.
I love how there are now more “intentions” – commerical, musical, physical under consideration. I suspect a new book may be in the offing?
What’s more exciting for me is the work on ActivityStreams which is capturing and formalising those intentions. http://www.activitystrea.ms
So whether it is (y)our Google Web History/Bookmarks, Buzz, Twitter, Facebook, Yelp or FourSquare, Blippy, Spotify, Nike+ etc. these will all be captured and made available as an XML/Atom stream possibly for further aggregation into a database of intent.
From these streams I think we will begin to see new personal tools for filtering and applying rules.
I also envisage new “trusted” companies forming whom we share our ActivityStream with for an exchange of value. These maybe in the form of discovery or recommendation – i.e based on my intentions find me new things or recommend me new things. Facebook and Google (Buzz) are going down this road.
The journey has only just begun.
Let’s step back and consider the more profound effects on people…
How soon will DoI data be used to judge who a person is — whether they are socially or morally acceptable?
Given how poorly we make such decisions today (based on outward elements such as manner of dress, or ethnicity) might DoI-based prejudice be an improvement?
What will we do to avoid misjudging a person based on mistaken identity? (Note often-cited jumbled credit histories, or TSA’s No-Fly list snafus.)
How could anyone defend against false DoI data being introduced by bad actors — out of spite, or any other destructive motivation?
As the volume of “bad” decisions grows, and everyone is miscast in some way, how will we rehabilitate our own identity? (There’s your $1G startup idea for ~2020: RehabMe.com — buy it now 😉
Since the “database” will be too vast and distributed to update, how will the errors of the past ever be corrected? (Otherwise, the next pass by any algorithm will promulgate the same error.)
Think “Big Brother,” with a perpetual, non-correctable memory. And, the “judge, jury and executioner” is already being implemented: http://www.computerworld.com/s/article/9164978/Narus_develops_a_scary_sleuth_for_social_media
I worked at NSA in the 1980’s. At that time, I would have said with assurance that NSA would not be associated with this activity. Now, though, I must ask: who will control all this?
I’m on the lookout for new Signals. I’m quite certain we’re not nearly finished creating them.
John, I can’t believe that you missed, as Matthew Hawn says above, the musical intentions. AudioScrobbler has been around since 2002 (http://www.audioscrobbler.net/), and integrated with Last.fm in 2004. It’s an old idea.
A more catholic definition of search would allow for a reconciliation of all these fields in the Database of Intentions.
And as nmw points out above, too, there already is a more catholic definition of search, one that has been around for 40 years. It is called “Information Retrieval”. What Google and the rest of the web search companies does is but a very, very small portion of what Information Retrieval has done for decades.
One signal to look out for is “What I’ve done”.
Some examples:
If you bought a computer 3 years ago, you might be due for a replacement.
If you bought an mp3, you might like to know about concerts.
If you booked airfare, you might want a car rental.
Phil
http://kartme.com/phil
My biggest “takeaway”..
If you’re not viewing your job to be a curator, clarifier, interpreter, and amplifier of the Database of Intentions, you’re soon going to be out of business.
Keeps me focused on what you need to be doing TODAY, even though you may not see the dividends immediately.
Thanks again for a great post.
Neil
I’ve always wondered how Google “Trending Search Terms” works. Still don’t know, but they were brilliant to come up with it – and now other search engines copy it.
Great synthesis once again. Even though i think you could leave the “Search” in favor of “Intentions” which are more generic and reflect a broader scope.
After reading the “Search” i tried map the “purchasing intentions” with the relevant Ad environment :
1) “Long term, potential buy” was generally well served by traditional TV/Paper Ads.
2) Mean term, rational buy, very well served by the Google ( Search+Adwords) Ads
And was wondering who would serve the last
3) “Short term, irrational buy” . Sometimes, we purchase things without any rational search, comparisons etc …but just as the mood takes you
3 years later, i think Twitter may be of good fit. It is real time, and most important feature for monetization : the notion of followers. Following someone means you trust their “taste ” and probably would buy what their recommend without too much rational thinking. (thus coming back to the immediate, irrational buy intention) just like stars sportswear product placements.
I think you’re right on.
I also believe that while this database of future intent is being collected by many companies , the integration of these different datasets will be important to really unlock value for users.
Search engines are not the only meta-players to be able to integrate these different datasets. Other datatypes that are may serve to define intent are:
i) purchase intent – companies like Yodlee, Mint, Blippy, CashEdge, credit card program managers etc. all have a view of actual individual transactions. Exposing this socially or for other app developers may be useful.
2) events/travel intent – companies like Plancast, Tripit etc. are starting to aggregate this data
3) health intent – companies like Fitbit, Dailyburn, Zeo, Withings are aggregating this data
You already mentioned location (Foursquare, Yelp, etc.) and search (google etc.).
While reading through it though I couldn’t help but relate most of your insights on the future of search, digital business models, and media more in synch w/ social networking than to traditional search. The database of intentions of today for me really is Facebook, not Google.
john,
great post. based on this posting from jeff pulver-
http://pulverblog.pulver.com/archives/009154.html
perhaps you should add a row under your “where i am” to indicate what gypsii does, which makes sense of all the above signals in a cohesive relevant mobile app, epitomizing the real time web. their idea of going beyond “search engines” to “connection engines” also seems to be the next big step.
thanks
This is true – but I’d add to it. I agree the “pure signal has been attenuating of late – overwhelmed by the sheer magnitude of data on the web, for one, and secondly by our own increasingly complicated expectations.”
And your speculations on other new “little signals” are quite compelling, but in the business world, I would argue that people are interested in different “little signals”.
The compelling little signal in the business world is generated by the detection of what’s changed – I call them “events” – which are patterns, connections and anomalies of interest to business decision-makers. “What has changed” is a powerful business signal. When detected by sophisticated search capabilities, it can transform the web from a database of intentions to database of actionable insights. For more on this, see http://www.huffingtonpost.com/penny-herscher/the-latest-web-search-dev_b_636854.html
Hi John
Great post. I loved the concept of a Database of Intentions.
Another signal, that I am involved in building right now is “what I know” – at wwww.bibkosh.com
Great synthesis once again. Even though i think you could leave the “Search” in favor of “Intentions” which are more generic and reflect a broader scope.
After reading the “Search” i tried map the “purchasing intentions” with the relevant Ad environment :
1) “Long term, potential buy” was generally well served by traditional TV/Paper Ads.
2) Mean term, rational buy, very well served by the Google ( Search+Adwords) Ads
And was wondering who would serve the last
3) “Short term, irrational buy” . Sometimes, we purchase things without any rational search, comparisons etc …but just as the mood takes you
3 years later, i think Twitter may be of good fit. It is real time, and most important feature for monetization : the notion of followers. Following someone means you trust their “taste ” and probably would buy what their recommend without too much rational thinking. (thus coming back to the immediate, irrational buy intention) just like stars sportswear product placements
The quilted jackets barbour with the fashion style is good. These juicy bag are able-bodied fabricated with the new design. The Juicy outlet do attending actual timeless.Therefore,these Barbour jackets for men attending altered from the accustomed barbour jacket in the market. The juicy backpack can wick damp abroad for your feet. You can get pleasure the absolute warmth with juicy couture. It adds abundant absorption to the Appearance Barbour jackets online.
I am so happy to read your post , it is wonderful .
http://www.bigcheaphandbags.com
Fantastic Read! Looking forward to more! Bookmarked the site and will be back again!
You leave so useful information,I will share this info with my friends.