The perfect search will be a combination of known selected information AND some pattern of collating constantly evolving additional information.
And it will point out not the perfect answer but a few sets of data which will comprise the 3 or 4 most likely answers.

Fragmentation:
Also as we evolve more and more with person or audience specific search we will have more and more fragmentation in vocabularies, thought processes, and ways of living.
This actually might be scary enough to bring us back to a more ‘utilitarian’ search paradigm rather than individual specific search.
Already it’s difficult to get people from a different line of work to understand/comprehend you well.

This is to some extent realized in today’s environment, but insofar as market forces and lack of imagination have led to the undoing of a lot of promising entities called “search engine companies,” we are unfortunately some ways away from even my very forgiving definition of “perfect search.” Danny Sullivan’s keynote in August seemed to be hopeful that we would be heading towards a “four-crawler” race (with healthy ongoing research at and consumer usage of Google, Teoma, Yahoo, and MSN) where things looked pretty bleak and one-sided for awhile there. But a four-crawler world is still behind where we were a few years ago, with at least five or six active web indexing companies alive at any given time. And the state of comprehensive human-edited directories is currently terrible. It, too, was better a few years ago.

There are a number of forces which could prevent us evolving into a relatively vibrant search economy (I say *could*):

- Monopoly — too few major search tech companies, too few web indexes, and no one seeming to care

- Privatism — good info if you pay a huge fee, bad info if you don’t, without a middle-ground low-fee option for the average person

- Standards (too few participants in some hypothetical future metadata scheme)

- Repression (governments)

- Commercialization via portalization — similar to monopoly; portals enforce a creeping, poorly-disclosed regime with too much paid inclusion and too many sponsored links

- Spam — same as above, but by cowboys

- Lack of imagination — if consumers and experts put the same old template on the idea of “what search is,” they may miss out on the idea that “a big list of keyword-search results, ranked based on a particular algorithm” is only one way of looking for stuff

- Illiteracy (too many people spend too much of their disposable income on 68″ TV’s, watch crap, and become stupid, thus no demand for “search”)

- Faddism in capital markets or gov’t policy — insufficient investment in search tools and basic research because something else becomes ‘hot’

In conclusion, I don’t really much believe in asking frivolous questions about perfect search. When Googlers are asked about it, don’t they just give the stock answer about the search engine reading your mind, and then quickly move on?

Is perfect search connected with access to information? It might be interesting to take stock of how far we’ve come in the past 100 or 500 years; to ask whether there are today any particularly premodern practices cropping up that fly in the face of trends towards fuller disclosure; and to consider how far the information revolution is likely to go. And probably it’s also worth considering that even in a society where there is a veritable smorgasbord of news and facts and information available, there are widely divergent ways of thinking based on different selection or different interpretations of facts.

The notion of “perfect search” is like the notion of “fixed ontology.” The way the world is seen depends on the way you categorize it. Those who advocate fixed categorizations of any kind (like arttworks, who admirably understands that global corporations are very powerful but cannot get past his interest in creating a fixed democratic alternative that might resemble a Stalinist regime) are suspect.

In a world where much information is privately held, one definition of perfect search would be to allow us to find out everything, to look under the veils of corporate and governmental power. If we live in a democratic society the trend must be in that direction, for sure. But these things unfold in shades of grey. I think China is a repressive society and would like to see some companies and some government agencies in the West disclose much more about their operations. But only a wide-eyed revolutionary would expect or wish every bit of information to be disclosed.

In light of that, “public” search engines perform a relatively limited service. Insofar as semi-private and personal information might become more available with the advent of “peer-to-peer” (etc.) search technologies (“hey, go ahead and search my desktop for the good of science or for the improvement of my own search experience”), privacy issues arise.

“Perfect search” works both ways. When the Big Spotlight is turned on you as a private citizen, private company, employee, or celebrity, it’s no longer so much fun.

More germane than perfecting search, whatever that could possibly mean, is finding an acceptable balance. Companies like Google are at a crossroads whereby they could make more money or create delightful new tools and features, just as long as they get more people to agree with creeping behavioral targeting and a slight loss of privacy. (GMail was one example of this, and there was a significant outcry. I thought the backlash was overdone, but the fears people have are not completely out of line.) Clearly some in the advertising industry (Claria) want to proceed full-out with behavioral targeting, spying, whatever you call it. Companies like Google have been much more cautious because they presumably believe that people like to be left alone and will punish the company that fails to respect that.

So on the privacy issue, there are already a number of “takes” on that, all of them active in the marketplace. Claria and Google alike have rights to offer tools and information in vastly different ways, instead of mostly frivolous lawsuits against them. That’s the postmodern let-multiple-flowers-bloom thing I’m talking about.

Future developments in search will grapple with complex socioeconomic equations, not just algorithms. We don’t build the perfect bridge, because bridges cause money and the perfect bridge would bankrupt us. We don’t run the perfect school, because perfect obedience is against children’s natures. But we do want better bridges for the money, and on average, we want better education for children (such as not teaching them things that are completely wrong or outdated, or making them inhale asbestos). How to get there becomes a complex policy debate.

In a perfect world, search would be moving in the right direction, but there would be no “ultimate” goal. Consumers and experts would evaluate and use a variety of information-gathering tools in such a way as to enhance their productivity and enrich their lives. Absolutes are not only dangerous, they’re silly.

In my own attempts to define what would make a perfect public or ‘open’ index (http://www.openindex.org/anidealindex.htm), I made up a list of features which I thought were desirable in such an index – what makes a ‘perfect’ index besides providing the right answer? With some modification, they include:

Complete – It would index all resources that are available to the public over the Internet.

Fast – It would be responsive and give good results quickly.

Timely – Up-to-date. It would be able to quickly discover and index resources and revisit them at appropriate times to record changes.

Relevant – Results from searches would be useful.

Accurate – The results would be correct and authoritative (I know: that’s hard!)

Honest and Fair – It would present search results fairly and without bias.

Private – Users of the index would be protected from surveillance and identification. It couldn

You’re probably familiar with the work of the TREC information retrieval evaluation series of workshops held over the last decade plus(http://trec.nist.gov), but if not, it’s worth repeating one of the most fundamental results that these people discovered. (Apologies – can’t remember the exact person/group and reference for this result off the top of my head. It’s likely to be Donna Harman or Ellen Vorhees however.)

Basically, in human judgements of information retrieval tasks, on average (measured over a number of queries – say 50 or more) two human judges agree on whether a document is relevant to a particular search query only 80% of the time!

In other words, for the exact same query, two people will disagree 2 out of 10 times whether or not a given result is in fact relevant. Without personalisation, it’s therefore nigh on impossible for a search engine to ever do better.

In John’s world of super-personalised search, the retrieval engine may be able to tailor its results to better meet the individual biases of the searcher, and bring them results which the searcher considers relevant, even if they would be irrelevant to another person.

But that said, in most TREC experiments, best performing search engines are mostly achieving an average precision (number of relevant results returned at a particular retrieval level) of only 40%.

Of course, there is a category of information retrieval queries for which one relevant result is sufficient (e.g. home page of John Battelle’s search blog), but there are a large number where this is not so.

All up then, there is a long way to go. But that’s what keeps it fun right?!