No need to repost it here, as my interview with Udi is up on Business2.com without subscription walls (thanks 2.0!). Includes an intro which gives an overview of the service and the implications. Good for those of you who don't want to wade through my last two posts on the subject...and gives a bit of insight into how Udi's mind works.

UPDATE: The link apparently is now behind reg, so the column is posted in extended entry. (6/21/04)

WEB EXCLUSIVE
Can Amazon Unplug Google?
The online retailer is planning to change the way we navigate the Web. An exclusive interview with Udi Manber, whose newly unveiled A9 engine just may drain the juice from the brightest light in search.

By John Battelle, Business2.com, April 15, 2004

Is Amazon (AMZN) taking on Google? Last fall the giant online retailer announced that it was launching a company focused exclusively on search. Lest anyone miss the implications, Amazon promptly rented an office far from its Seattle headquarters -- in the epicenter of Silicon Valley, near Stanford University. That's where both Google and Yahoo (YHOO) were born.

Tapped to lead the startup was Udi Manber, Amazon's chief algorithms officer and Yahoo's former chief scientist. Widely respected for his elegant approaches to intractable software problems, Manber was a magnet for Silicon Valley talent. (Either in jest or retaliation, Google responded by buying advertisements on its own search engine for the phrase "Udi Manber"; type his name into the search form and the top ad link offers job opportunities at Google.)

Amazon recently took the wraps off its initial public beta version of A9, and we got the first sneak peek. It's easy to see why the new engine might set Google and other search-driven companies on edge: A9 is a credible step toward making a search engine that knows you and acts as your agent online.

By overlaying what you're looking for onto what you've already found -- and, ultimately, what you've consumed -- A9 aims for the holy grail of search and e-commerce. While others have tried to create personalized interfaces, none has worked so far. Users don't like to answer questionnaires about what they like and what they don't; often they lie. But Amazon already knows who you are -- or at least, what you buy. With A9, the company can factor in what you browse and search for as well.

And the best part, from Amazon's perspective, is that the more things you buy at Amazon, and the more you browse with A9, the better the company will know you. Ideally, it's win-win -- you get better results, Amazon gets more sales.

Manber is the first to tell you he's nowhere near his goal of creating the perfect search engine. But A9 -- the name stands for the nine letters in the word "algorithm" -- is a significant evolution. Blending powerful personalization features with clever interface tweaks and a truly useful toolbar, A9 could well do Google one better. Interestingly, it uses Google's search results -- Amazon and Google have a longstanding search deal -- but builds on them with an elegant set of features that, taken together, make A9 seem like a new way to search.

In truth, Amazon and Google are speeding from two ends toward a very fertile middle ground where commerce and search meet. When they get there, they'll have formidable company -- AOL, eBay (EBAY), Microsoft (MSFT), and Yahoo, to name a few. All of these companies have millions of loyal users and business models built on connecting those who seek with those who sell.

And there's an obvious question: Does Amazon honestly think Google will stand by while Manber creates a better way to search?

It's clear he's just getting started. A native of Israel, Manber has an almost elfish grin and the merry intensity born of someone who clearly knows more than he's allowed to tell. John Battelle asked him a lot of questions anyway.

Walk me through what's different about A9 as a search site.

When people come to A9.com, they have the option to install a browser toolbar and register through Amazon's regular registration process. If you're already an Amazon customer, the site automatically recognizes you. If not, you don't have to register, but if you do, you get the personalized features, and if you install the toolbar, you get even better services.

As to what is new, the most obvious feature is your personal search history -- which is integrated into your entire search experience. So your entire search history is available to you, and with the toolbar, that includes all your searches across any search site, as well as all your browsing on the Web. Also, all the interface columns on the site -- personal history, or related books, or the Web search column itself -- are adjustable and the site remembers the settings the next time you come (the default is that Web search only is opened). I'm very proud of our toolbar's diary feature that allows you to annotate each site you visit. And we have integrated Amazon's "search inside the book" feature into the engine, so now all your results include excerpts from related books.

I understand much of the personalization is made possible by what you call a "history server." What exactly is that?

The history server stores -- on our servers -- your history of interaction with us for the purpose of bringing that back to you in a very convenient way. Whenever you come to the site, we can show you what you searched for in the past in a very easy-to-organize fashion. If you want to hide some of that, you can opt out at any time. If you install the toolbar, then all your Web browsing, as well as all your searching, is stored as well. And we are working on many different ways to improve that.

So there is a massive repository of data, held on an Amazon server, that tracks where I've been on the Web. Isn't that something of a privacy nightmare?

Our privacy policy is very clear on this subject -- we will never share this data with third parties. Also, having the data of what users search for or where they go is not a new concept. Any site that requires registration -- AOL or MSN, for example -- already has this information. What is new is that we're taking this information and giving it back to the users to make their search experience better and more useful. It gives them more power.

Will A9 ever integrate all the information Amazon already knows about its customers -- what they buy, browse, or search -- into creating better search results? If I happen to buy a lot of books on jungle cats, will my A9 search results know to serve me the cat-related search results for the query "jaguar" ahead of the car or the football team results?

We have no plans to do this at the moment. It is very hard to do well. If we do this, we want to make sure to do it in a way that makes the search experience work better for users. But again, I can't predict the future. The whole goal here is to get better customer experience. Whatever I can find to get better customer experience, I'll do it.

How can we square the perception that A9 was going to be a commerce engine with the reality of the site as it debuted, which feels much more like the evolution of a pure search site like Google? Will a shopping comparison engine be added soon?

I can't answer that. As an engineer I don't like to talk about things that I haven't yet built. A9 is a way to experiment with several features that help our users. It's a way to bring more users in, and it's a way to see the feedback they give us using those features. We will later use that feedback to improve search both at Amazon and other sites we partner with. It is clear that search is an integral part of any e-commerce site. It's also clear that search is not a solved problem for e-commerce, or beyond e-commerce.

What happens if A9 starts to steal market share from Google?

We don't talk about other companies, and to be honest, we don't even think too much about competitors. We try to concentrate completely on the users. I don't look at it as who we compete with but rather how much something helps the users.

You've said search is not solved. What are the problems that need to be addressed?

In general, the main problem of search is very simple: Too often people do not get the results they need. Solving that is hard. We cannot read people's minds, but we can do much better anticipating their needs and making the process much easier for them.

What might we expect from A9 in the future?

A9 is going to roll out features over time. This is our first beta test -- the result of a small team working over a short amount of time. We are growing the group as fast as we can. We have a large queue of very good ideas with the purpose of making search better. Our long-term goal is to improve search. We hope to invent new search technologies and use them on Amazon.com and other partner sites.

John Battelle is a visiting professor at the University of California at Berkeley and author of "The Search" (Portfolio, late 2004).

Find this article at http://www.business2.com/b2/subscribers/articles/1,17863,611251,00.html

©2004 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.