Danny and I have contracted with Thomson Reuters, a sponsor of Searchblog, to write a series of posts on the future of search. They’ve given us no guidance, just asked us to ponder the topic. This is my first post, “A search is not just a search,” longtime readers will find it familiar, if updated. From it:
In the past few years, a significant new feature has crept into the results portion of this otherwise predictable interface. Called “universal search,” the idea is to incorporate more than simple HTML pages into the results. A search for “London restaurants”, for example, might bring up maps and local results, as well as videos, images, organized reviews, and of course web pages. Every major search engine, from Google to Ask, has incorporated some kind of universality into its search results.
But while universal search points the way toward a new approach to getting you the answers you seek, it’s a half step at best. The results change, somewhat, but the process is pretty much the same. You enter a query, you get a set of results. Not particularly new.
What I find interesting are entirely new approaches to the interface of search.
We’ll be writing one post every week or so for the next six weeks. I hope by the end it’ll be an interesting body of work, it certainly will be if you give me input on what to think about, and critiques of what I’ve written. Thanks!
Yahoo! India added it too..
Check out http://in.search.yahoo.com/search?&fr=fpa_onnetwork_fp_mast&ei=UTF-8&rd=r1&p=sachin+tendulkar
As an Italian I liked your previous “future of search” example (how many years ago?)
“have a device which is in your pocket, which looks like a phone, and you go to a supermarket and you see a potentially overpriced box of pasta. And you take that device and you wand it over the product code, and you see comparison prices from Google of three other stores that are within a mile, OK? That’s power. That’s search. But no one has quite figured out that. That’s also the future.”
I find cool search tools often. But no matter how cool they are, it is still very uncool to get dumb results. Something has to be done to get better results. Everyday I use Google so much time is wasted on stupid pages with very little if any relevance. Google is getting to be about as good as and old yellow pages.
Some of the best search tools I’ve found recently are PicLens (ref: readwriteweb.com) Really amazing search tool, very apple-ish. ModernWeb (ref: woodymaxim.com) Very very thorough search tool, and Viewzi (ref: the agencyblog.com) Another very unique tool.
But like I stated before, the search results have got to be relevant. It can be the simplest search engine in the world, and if the results are relevant it will be golden. Even more so than Google.
Tony you hit it on the head. Google results are Blah. They have little incentive to improve because that would mean less add clicks. The search genre is wide open for the taking for whoever delivers consumers custom vertical results that are human generated. Nobody really cares about these algorithm based search engines static text results. Strategic Multichannel Vertical Location Social Networks are the future to online destinations. In my opinion custom social netwoked user generated results that combine word of mouth marketing with pinpoint standardized vertical location is what consumers want. Custom personalized Location Networks and results are the future of search.
While I agree with Tony that the results must be relevant, a key factor here is that often the user does not know ahead of time what is relevant. Searches are often performed not to find a specific web page or a specific product, but to come to an understanding of a certain topic area. Once enough information is discovered, then the user is able to form a real question, and come up with a real information need.
Currently, search engines do not support this type of behavior at all. Search engines are geared toward getting you to a specific web page or piece of information.
Search engines need to evolve to take into account.. well.. the evolution of user information seeking activity. Search engines need to aid the information exploration process, not just the home page finding process.
“Strategic Multichannel Vertical Location Social Networks”
Steve, now that is a mouthful. Custom personalized location networks truly does sound like the future. A social network search engine that delivers what I need the first time, absolutely genius!
From your response, you sound like a CEO of the next generation of search companies.
Sincerely,
Tony Edward
Respectfully dissent from your choice of vineyard and from your view on the future of magazines…..
http://exacteditions.blogspot.com/2008/05/future-of-search-and-future-of.html
Come to think of it, if you could do one you could do the other. I quite fancy the idea of subscribing to a digital wine with archived and digitally ‘remastered’ vintages…….
JG & Tony,
if you review http://battellemedia.com/archives/003991.php , you may notice that many (indeed, perhaps even most) people use Google for “white pages” lookup.
In other words, people do not generally use Google to find something they don’t already know about. “New information” (actually somewhat of a redundant tautology) is found elsewhere — and Google is primarily used for “known item searches”).
I’ll never forget the way some librarians in the old days of paper card catalogs proudly smiled that “their” catalog could be used to find reliable information such as the birth year and/or year of death of an author (and therefore no additional resources were needed).
Similarly, Google can be used to “find” the url of a domain name (in other words: you can use the space bar instead of actually using the “dot” key, and then the top result will normally have the name with the dot) — I agree: not a great feat, but as long as browsers require people to use the “dot” (“period”) rather than the space bar, it may still be a millisecond or two faster than typing in the correct URL directly. In some cases, when brand names are well secured / protected, the TLD extension can also be left off — and then the time savings may be even greater (perhaps half a second or more!)
This is the primary reason Google is useful: It has little to do with discovering anything new, but rather it’s mostly about saving the effort of having to type out brand names.
Search in the sense of “discovery” is usually done elsewhere (if it’s done at all) — e.g. “vertical search engines”, such as baby.com (and/or similar “targeted” keyword domain names). The main reason why Google is hesitant to divulge the stats of the amount of money it earns from such domains (parking pages) is that they are afraid that people will realize that Google’s search engine is so totally ineffective that it’s laughable that anyone would waste their time and/or money by giving it to Google.
Sooner or later, people will catch on, and then Google’s share price will really tank big time….
Interesting. I’ve presented on the Future of Search a few times and the more I get into it the more amazing the number of opportunities become. As Larry Page said a few years ago we’re only 5% of the way there, since then I reckon we’ve moved barely a couple of percentage points further forward. Coincidentally MarkE just covered a future without links on our blog. Feel free to quote.
http://www.search-engine-war.co.uk/2008/05/the-future-with.html
nmw: you may notice that many (indeed, perhaps even most) people use Google for “white pages” lookup.
Oh, I notice that. Believe me, I’ve noticed that for years. And ranted about that for years. I’ve asked Google, publicly in forums like this, privately to their engineers, semi-publicly at conferences, to get over it already, and get on with the business of organizing the world’s information by doing more than just known item search.
They’ve refused to budge.
That’s why I find it laughable when Marissa Mayer gets up there and talks about how they’re improving their search engine all the time. About how there were like 450 changes (improvements) to the search engine last year alone. So what? 450 changes to an engine that does known item search is.. still an engine that does known item search. And nothing else. Nothing has changed. Google is giving me no new insights, no new ways to help organize the world’s information. 10 years into it, Google’s still only 5% of the way there, as Teddie above mentions.
You continue: In other words, people do not generally use Google to find something they don’t already know about.
Do they not use Google because they do not want to use Google? Or do they not use Google because Google is not providing a search engine that meets their needs?
I would argue that it’s only 50% of the former reason, and simultaneously 100% of the latter.
What I mean is this: Go back and read Andrei Broder’s 2002 paper, “A Taxonomy of Web Search”:
http://www.sigir.org/forum/F2002/broder.pdf
In it, he makes the case that Web search is not like traditional IR.. that in addition to “informational” queries, web engines need to be able to answer navigational (known item) and transactional queries.
Fine. That’s all good. Kudos to the big web search engine companies for finally bringing us good navigational searches.
But what this paper also says is that between 39% (from a user study) and 48% (from log analysis) of web search engine queries are informational, not known item.
This means than more than a third, if not closer to a half, of every single query running through Google’s engine is a query that Google has specifically designed its search engine to not be able to answer!
That’s completely ridiculous.
(Conversely, it says that between 52% and 61% of all queries are NOT informational, which is why I do concede that not every person, all the time, is doing an informational query. But it’s also not true that no one ever does informational queries! 1 out of every 3, if not 1 out of every 2 queries is informational!)
So I’ll say again: Search engines need to evolve again, and get beyond doing navigation-only searches. Look at what John Battelle wrote about it 5 years ago:
http://battellemedia.com/archives/test/000100.php
John says that search engines are moving from navigational to transactional. But that was 5 years ago, and they appear to have stopped moving. Almost 1 out of every 2 queries, informational queries, passing through Google are not only not being answered, they are not even attempting to be answered.
Sooner or later, people will catch on, and then Google’s share price will really tank big time….
I think Google suffers from a second problem in this space, too: The innovator’s dilemma.
Because Google so heavily aligned itself with doing navigation-only queries, to the complete detriment of the 1/3 to 1/2 of the informational queries passing through its search box, it has succeeded in turning itself into an AdWords money-printing machine.
Think about it: Google is trying to condition people to think of it only as a navigation/transaction engine, only, rather than a true “search” engine. And the reason it does that is there is a higher chance of people clicking on ads, if the intent behind the user of Google navigation/transaction engine is to find a place to buy something.
A user arriving at a search engine with an informational query is not going to spend much, if any, time clicking on ads. Because what ads have to offer are not the intent or goal of an informational search!
Let me say that again, because it is one of the biggest conceits that we are collectively operating under in this industry: If someone has an informational query, an ad is (almost) never going to satisfy their information need. Ads presuppose a navigational or transactional query type!
So the innovators dilemma for Google is that if they actually altered their search engine to be able to handle informational queries, two things would happen: (1) It would require more effort from Google, both in terms of algorithm designers and programmers, as well as CPU usage, memory, disk space, bandwidth, etc. Informational queries are more computationally expensive! So costs for Google go up.
At the same time, (2) ad clicks would go down, because advertisements tend not to be relevant to informational queries, as mentioned above.
So in order to innovate (provide good algorithms and interfaces for informational queries), Google would need to destroy the very business that got them where they are today (navigational/transactional queries, laced with ads).
That’s another reason why I cringe every time Google says its ads are independent of its search results. They’re not. Google consciously avoids developing certain type of search results and search result interfaces, because that would interfere with ad click throughs.
Hi John,
I have been reading this blog for ages and have never felt cause to comment until this post.
I’m not sure where I first heard the phrase but I always have “the Star Trek model” in mind when thinking of the future of anything. For example, if the holodeck is one part of our future, then online virtual worlds and (to a lesser extent) social media are predictably popular as harbingers of the holodeck experience.
In terms of search “the Star Trek model” dictates the ability to call out a question (preferably prefaced with the address, “Computer!”) and have a disembodied voice answer that question with a word or a graphic or holographic display and whatever it deems relevant. Built into this answer seems to be an understanding of the person asking and a pile of other considerations. As well, the questioner will usually still have some understanding of the need for context and the limitations of the computer. As stated a few times above, we’re only 5% of the way there.
Even better than the Star Trek model will be the lack of any need for search: how will we get to the point where we will not need any questions answered.
John – we’re currently running a research project, results will be freely and publicly available at the end of June.
Topic is Findability, and it’s focused more on inside the Enterprise than outside, but still, might be of interest to you and your crowd here.
Closing the survey down next week, but can always use more data. >400 responses right now, but would love to raise that significantly to get an even bigger picture.
Project is described more at:
http://www.biztechtalk.com/2008/04/findability-sur.html
And the survey itself is live at:
http://aiimMarketIntelligence.questionpro.com
Average time to complete the survey is between 20-25 minutes at this point.
Participants have a chance to win 1 of 25 gift certificates at Amazon.com, as well as early notification of the free finished report, and companion free public webinar discussing the results, both available the last week of June.
I’ll second the ongoing discussion here – while frequently people use search to find “known information” there are plenty of times when people need to discover new information, and be more adequately lead in the direction of useful “leads” on such information.
That’s why our survey is on Findability rather than Search as the umbrella term. What about taxonomies? ontologies? relevancy ranking algorithms? dynamic clustering? impact of social interaction with information back on relevancy, discover, etc.?
Text retrieval (search) is over 30 years old, and yet we’re still barely taking advantage of what could be done – not just from a “wow, that’s a great demo” but from a true, “made a difference to my business” aspect.
I realize the enterprise angle is not your primary focus here, but certainly relates. How do consumer, web-focused search solutions address the differing needs of the enterprise? Does one corrupt the other? What do both sides have to learn from each other?
Interesting times!
Cheers,
Dan
http://www.twitter.com/dankeldsen
>> This means than more than a third, if not closer to a half, of every single query running through Google’s engine is a query that Google has specifically designed its search engine to not be able to answer!
I made http://browse.name in a couple hours like 3 or 4 years ago (well, that’s what it looks like, too — I guess back then most of the time was just to upload the framesets 😉 It’s pretty crappy and quite outdated, but probably still 10x better than hoping to find information using Google.COM ;P
However: If Google worked more *like* browse name it might function at least a *little* better (there was someone who posted here on battellemedia.com about half a year ago who is apparently developing such an engine, BTW).
The basic idea is that hotels.com is actually not merely a “navigational” result (when searching for “hotels”) — hotels.com actually delivers information about hotels (and in particular: commercial hotels)). But I don’t need google to know that. Just like Spock on Star Trek, I can simply find “logical” answers: hotels.de is about hotels in Germany, hotels.be is about hotels in Belgium, hotels.no is about hotels in Norway, etc.
So the point is that Google is simply helping out the novices who haven’t figured out how to use their browser to find information — at least in theory. In practice I think you are right: Google is not interested in producing any useful results whatsoever, since that would negatively affect the immediate revenue streams.
The main issue is, however, that Google pretends to have a “secret formula” that magically returns hotels.com when using google.com (and that returns hotels.de when using google.de) to search for “hotels” — and that’s why they need to have 15,000 Ph.D.s working on it. So far, a lot of people actually still believe this! :O
;D nmw
Here’s one such example of the engine I describe above:
http://searchdns.netcraft.com/?restriction=site+contains&host=tennis
Now if I could search the content within these domains for “racquet”, then I think Google would start to look very old school.
I believe that the web has changed since the first search engines appeared on the scene. The web was then a collection of hyperlinked documents. The web is now a collection of applications, such Facebook, Yelp, Twitter, and so on. Search engines have to “think outside the index” and evolve into routers of search queries to applications.
More in this post:
http://anand.typepad.com/datawocky/2008/05/why-yahoo-glue.html
While researching a real need to improve candidate and resume search within our product – http://www.TargetRecruit.net, I stumbled into your book “The Search”, and I am thankfull that your book gave me good clues of good, bad and ugly part of Google Search and made me think through our own strategy for search.
Our dilemma is introducing “relevant” search within business application – which are driven by more than just associated quality links, but also ratings, meta-data and resume match to a job amongst others. The current world of resume search largely depends on “sql” query – I think the future will be a combination of many more data elements like feedback, relevancy, background and many more meta data for which algorithms would need to be developed to matched appropriately. This problem can easily be extended to other business place applications like vendor management, marketplace ratings and recommendations etc. We want to go beyond traditional queries to intelligent relevancy
One of the story in your book resonated close to home. We have another product DBSync for Salesforce.com and QuickBooks which is one of highly recommended and downloaded as compared to our competition. We had listed it on Salesforce.com Marketplace – AppExchange and it used to rank quite high on “QuickBooks” keyword as its initial “relevance” algorithm seemed to handle other meta data. Then suddenly the ranking fell on its “Relevance” page, and you had now a product listed on top which had no review or rating and limited functionality – all other “relevant” applications fell way below. When I checked, they said they had switched to “Google Search”.
The above begs the discussion that how misunderstood search is and how un-planned organizations are while pursuing their search strategy and implementations. Salesforce.com AppExchange would have been better off following the Amazon model of recommendation and search – which I feel is more relevant in such situations. Perhaps at some point you might want to write another book for Amazon Search…
While I will continue my research on implementing search within our application, which will not likely be Google, I will certainly look forward to your blog for ideas and suggestions.
Rajeev Gupta
CTO, Avankia