Friday Book Excerpt: More on Perfect Search

Long time readers may recall when I posted a thread on perfect search a year or so ago. I got so many wonderful answers, and I wove much of what I learned into my final chapter. This is a short excerpt from that chapter. Again, please send or post comments, corrections, clarifications – I already see a few things here that I would like to tweak…as usual…a book on topics like search is a never ending work…like a blog!

Search Everywhere

In the near future, search will metastasize from its origins on the PC-centric Web and be let loose on all manner of devices. This has already begun with mobile phones and PDAs; expect it to continue, viruslike, until search is built into every digital device touching our lives. The telephone, the automobile, the television, the stereo, the lowliest object with a chip and the ability to connect—all will incorporate network-aware search.

This is no fantasy; this is simple logic. As more and more of our lives become connected, digitized, and computed, we will need navigation and context interfaces to cope. What is TiVo, after all, but a search interface for television? ITunes? Search for music. That box of photographs under your bed and the pile of CDs teetering next to your stereo? Analog artifacts, awaiting their digital rebirth. How might you find that photo of you and your lover on the beach in Greece from fifteen years ago? Either you scan it in, or you lose it to the moldering embrace of analog obscurity. But your children will have no such problems; their photographs are already entirely digital and searchable—complete with metadata tagged right in (date, time, and soon, context).1

But let’s not stop our digital fantasy train yet. It may sound farfetched, but in the future, your luggage will be searchable. Within two decades, nearly everything of value to someone will be tagged with tiny computing devices, devices capable of saying, upon radiowave-based query, “I’m here, right here, and here’s what I’ve been doing while you were away.” Instead of the ubiquitous bar codes airport officials now slap onto your luggage, there’ll simply be an RFID (radio frequency ID) chip. Lost your luggage? I don’t think so. Not when you can Google your Louis Vuitton in real time.

Think about that—Google your dog, your kid, your purse, your cell phone, your car. (Do you have an E-ZPass or OnStar yet? You will.) The list quickly stretches toward the infinite. Anywhere there might be a chip, there can and most likely will be search. But for perfect search to happen, search needs to be everywhere, attached to everything.

This means that among many other things, search needs to solve what so far has been a rather intractable problem: that of the invisible Web. As Gary Price and Chris Sherman point out in their book of that name,2 the invisible Web comprises everything that is available via the Web, but has yet to be found by search engines. Deep databases of knowledge, like the University of California’s library system or the LexisNexis news and legal citation service, are walled off from search for commercial or technological reasons. And while the contents of your hard drive may be digital, they most likely have not been indexed and offered up to a search application—yet. As I pointed out earlier, all the major search engines have launched desktop search tools which index your hard drive and serve up the results much as you might see Web results. Prior to the advent of desktop search, your PC was part of the invisible Web. No longer.

Also mostly invisible, and mainly still stuck in the analog world, is what might be called the content Web. There are nearly 100 million books extant, but only several hundred thousand online as of this writing. Add to that unsearchable pile humanity’s analog archives of film, television, and periodicals.

Thanks to Napster, we’ve already got the music nut partially cracked. When Napster launched, millions of people ripped copies of their favorite music to the Web. And therein most likely lies the solution to the rest of our previously unsearchable media. For nearly every book, film, and television show, someone, somewhere, will come up with a reason to put it on the Web, assuming we can get out of our own way with regard to intellectual property issues.3 Massive archiving projects, such as Google Print, the Internet Archive, and Amazon’s Search Inside the Book, have gone a long way toward solving a piece of this problem, but they have a long, long way to go, and simple logic tells us that no one entity can (or should) archive the sum total of humankind’s information. No, when it comes to making the world searchable, the best way is to simply let the world do it.

This phenomenon has many casual monikers, but I like to call it the Force of the Many. Eventually, everything of value—including your luggage—will be connected to the Web, because to be connected is definitional to the concept of value in a wired world. As the Force of the Many weaves humanity’s belongings into the Web, search engines will weave this new content into their indexes, moving the world ever closer to the possibility of perfect search.