John Battelle's Searchblog

No doubt this project was hastened by Google's new approach to travel search. From the release:

Mobissimo One-Box Search technology allows customers to find online travel deals even more quickly and easily as it eliminates the need for customers to check multiple boxes, click calendars or type text in forms to obtain the best fares and rates for airfare, hotels and car rentals. Mobissimo’s One-Box Search processes information in just the way that consumers think. The user merely types in the departure city, the destination city and the dates.

Has the worm turned? That's the question a lot of folks are asking about Google these days, not the least of which is John Heilemann, whose piece has opened up a great discussion in the comments area of my post about it. As we often do, John and I beat this question around a bit this morning, and an interesting comparison came about. John, who wrote an excellent book on Microsoft called Pride Before the Fall, reminded me that while most peg Microsoft's fall from its glory days to the attenuating DOJ trial of the late 1990s, the company's true fall from grace came before that trial, when first the digerati, then the company's potential partners started losing trust in Microsoft.

Why? John pegs it to a seminal 1997 Wall Street Journal article about the company in which Nathan Myhrvold (former MSFT CTO) speaks of his company taking a "vig" on nearly every transaction across the Internet. A year or so before that article, while managing editor of Wired, I met with Nathan. He pitched me his vision of Microsoft enabling - and profiting from - all commerce on the web (I wrote the meeting up for HotWired, but can't find the damn link...). In any case, I recall Nathan taking out his wallet and slapping it on the table, and confidently predicting that anything you did with a wallet, Microsoft would own online. I was struck by the arrogance of such a claim, and the confidence with which he made it....I really believed that Microsoft was going to own ecommerce, and it both scared and fascinated me. Turns out, I was not alone.

As we discussed the finer points of the AAP lawsuit, John noted that Google is coming close to a "worm turning" moment - a moment when the world realizes that the company is *too powerful* and its ambitions are *too great.* When such a genie arrives, it is very, very hard to put back in the bottle. The one all encompassing difference, of course, is that Google has real competition - Microsoft in 1997 did not - but regardless, the cultural vibe is striking in its similarity. Remember in 1995, when Microsoft was literally at the top of its game, lauded on the covers of national magazines for saving the US economy via its launch of Windows 95? When Gates and Co. were heralded as ushering in a new era of digitized possibility?

I sure do. In seven short years, Google has gone from a geeky startup with one good idea into an agenda-shaping player responsible for navigating complex relationships with world governments, the personal privacy of millions, major trade organizations, and hundreds of thousands of businesses small and large. It's an extraordinary weight to bear, it seems to me. It's the kind of position that requires a balanced mixture of leadership, will, and diplomacy. There's very little room for the go-it-alone mentality which got the company to where it stands today. Can the company shift its culture and avoid the fate which ultimately hobbled Microsoft? That, more than anything else, will define the next chapter in the company's fascinating story.

Raul Valdes-Perez of Vivisimo begs to differ with all the hype around personalized search (including in my book), and the idea of major engines mining your clickstream to better understand your intent (and give you more personalized ads, of course). In a short paper outlining his views, (PDF download), he outlines five major problems with personalized search and concludes:

.... search personalization is likely to waste the talents of top computer
scientists. It may even give worse results...

Google is accelerating the pace of change in both its results and its home page. Many readers have emailed me recently with tips, here are a few:

First, "Personalized Home" is now standard on the home page, up on the top right. This is an attempt to push more of us toward using its portal Fusion.

Also, many have noticed the new way Google is clustering results based on similar phrases. Try searches for President's Day Weekend, for example. And lastly, as I've noted before, Google continues to push adoption of its Toolbar, now with banner ads for it at the bottom of results (thanks Peter).

As reported by PaidContent, Halsey Minor, founder of Cnet and Grand Central, has recapped GC and turned it into an online advertising/campaign management company, Swivel. More here.

Google appoints Hollywood vet Ann Mather to its board. Mather is a financial expert from Pixar, Village Roadshow, and Paramount. Perhaps she can convince Hollywood to put some of their eggs in Google's video basket....

For his New York column, John keys off the AAP lawsuit:

The signs are everywhere. In France, Jacques Chirac has ordered his minions to gin up a French and German search engine—on the grounds that Google is (wait for it) a tool of U.S. cultural imperialism. In Bentonville, Arkansas, Wal-Mart board members admit to keeping a wary eye on Google—whose capacity to alert shoppers to better bargains elsewhere is seen as a burgeoning threat. Even out in Silicon Valley, reproachful accusations are hurled that the once-beloved leader of the Internet resurgence has taken on a dark Microsoftian cast.

He quotes AAP Pat Schroeder:

“Alan Murray wrote a column in the Wall Street Journal that called Google’s business model a new kind of feudalism: The peasants produce the content; Google makes the profits,” she informs me, then ladles on an extra helping of ominous foreboding. “Do we really want one corporation controlling all the content in the world?”

Then explains how the case turns on interpretation of fair use:

Who’s right? Impossible to say. By all accounts, the law of fair use is mind-bendingly complex: “There are parts of it that I don’t understand, and I’ve been studying it for years,” says Lawrence Lessig, the Stanford intellectual-property guru. Like virtually everyone involved in the dispute—he’s a vocal Google backer—Lessig allows that there are precedents that point in each direction. But he also acknowledges that the legal issues are in some respects peripheral, for the battle is actually being fueled by factors at once more venal and more visceral.

And then gets into the real business at hand:

If Google were to stick to its pledges about how it would employ the megadatabase of books that it’s constructing, the book business would likely benefit. But publishers don’t believe that Google can be relied on to keep its word. They fear that the company, which has made a mint off a technology, the Internet, that publishers still only vaguely comprehend, will someday abandon its putative adherence to just-the-snippets fair use and screw the publishers with their pants on.

As usual, a fun and worthwhile read.

According to Om, Boing Boing is worth $34 million. Who knew?

Full article here. From it:

Based on recent high-profile Web content deals, the value of a unique monthly website visitor currently hovers around $38 (the average purchase price per unique user of acquisitions during the past year). As a result, those who built popular websites over the last few years look prescient: They “bought” eyeballs when the market placed little value on them -- making daily blog posts or encouraging others to upload text and photos -- and can now sell their traffic at a markup.

While I don't doubt the math - there are plenty of comps and Om has 'em all in the piece - I'm not convinced. Of course, I have a stake in all of this, what with FM and my involvement with Boing Boing. But we've been around awhile, and so far not many folks have come offering thirty million plus. Why? I'm guessing because buyers are smart enough to realize that Boing Boing and sites like it are unique voices with fierce attitudes about content and the author/audience relationship. It'd be pretty hard to buy that and simply shove a big dancing flash ad in there and make your investment back - you'd lose the very thing you bought it for - the audience. It's harder, but ultimately more profitable, to run sites like this as smaller, profitable businesses for a while, at least until the marketing approaches emerge that are accepted and embraced by author, advertiser, and audience alike. That takes time and experimentation, not a rush to sale. But it sure is fun to dream....

The bricks keep getting laid in my TV/Search scenario. The WSJ (paid reg) (update free link is now avail) reports that Tivo is partnering with several large agencies to bring topic-specific search to TV advertisements.

From the piece:

People who watch traditional television are forced to view commercials in random fashion, regardless of what they may be interested in buying, says Tom Rogers, TiVo's president and chief executive. "We're flipping the dynamic," he says, allowing TiVo subscribers to search for ads that match their interests. "If you are in the market for a product, and you have no idea when commercials related to that kind of product are going to appear, it doesn't help you very much," he adds.

TiVo users will be able to set up a profile of products on their television screens by clicking on categories such as automotive or travel or typing in keywords such as "BMW" or "cruises." On a regular basis, TiVo will then download relevant commercials to TiVo recorders over the Internet or, for those users who don't have broadband, send the video via traditional broadcast signals. The commercials will appear on-screen in a folder next to the list of television shows TiVo users record.

Advertisers, in turn, will be able to select the keywords and categories with which they wish to be associated for their ads. TiVo is in discussions with advertising agencies about the best way to price such advertising, but one option is to let advertisers bid on keywords as they do when buying ads on Internet search engines.

The Search has debuted in Japanese. Editions are coming out in around ten countries, so far. I sure wish I could read Japanese so I could see how the book is selling...

This is not really search related, but...in a broader sense, in fact it is. Read this link. What do you think? If you think the woman should have to show ID when some low level bozo cop boards the bus, then prepare to have your entire clickstream similarly demanded, on equally flimsy pretense. Scary. I hope she wins.

A fun "blind taste test" between Yahoo, MSN, and Google is here. Really interesting.

Google is starting to poke around the pay per call market, and the company has also begun accepting local merchant info into its Froogle application. Whether local merchants will list is an open question, for more read the wine anecdote.

Google also announced a $3 million gift to the LoC for its digital library project. The Print/Book Search issue is really not going away, by the way, and remains a seminal debate. More to come on that, interesting to note folks changing sides on the issue...

Ask and GoFish announce multimedia search deal.

Via Silicon Beat, Pandora, a music search site, takes in $12 million. This feels like a hell of a lot. Also from SB, Dipsie, which has been very quiet for a year, has launched with a different, deep web model.

Dick Costolo is thinking hard about how to make RSS better. Read this post. (And this one from Fred).

Watch this site: Xooglers, the blog of Google's former Director of Consumer Marketing Doug Edwards (he left after five years, he acknowledges that "my life is good"). The posts are fascinating. From one of them:

It's a long story, but one I now have lots of time to tell. This blog is partly about that, but mostly about what happened during the following five years and three months, while I served as Director of Consumer Marketing and Brand Management for Google.

For the last eight months, I've been gathering my thoughts in preparation for writing a book. That may still be forthcoming, but the more I think about it, the more convinced I become that a book would not be the Google way to do this.

And another:

Your S.A.T. score was the measure of your intellectual capability; your GPA represented the numerical summary of your ability to execute on that potential. Your value to Google could be plotted using those two data points.

Sergey's desire to reduce every decision to an equation would cause me a fair amount of frustration in the years to come. While it forced a discipline on me that was likely lacking in my career up to that point, it also went against my deeply-held conviction that some things are not expressible simply by deriving the correct algorithm.

Businessweek gives Google the cover (surprise!) with two stories, one which focuses on the company's odd approach to business development and M&A, and the other which focuses on the culture of wealth within the ranks of the employees. Despite the cliche of covering (literally) Google a bit too obsessively, each story does break ground. Worthy reads.

Watch this space....

In other words, you can now bid just for Google.com ads, or for content-driven ads from AdSense. There was a lot of confusion in the marketplace prior to this, now we'll see the two products really evolve independently. MediaPost coverage.

A long flight from London. And lots of news. Will post a digest before Turkey....hope all of you have a great holiday, or if you are not in the US, that you bear with us as we take stock and give thanks...

Alas, The Search did not win the FT Business Book of the Year award. But I'm happy to be in the company of those who were shortlisted, and Thomas Friedman, who won. News first came from Pravda, oddly, but I was there, and it was a fun night nonetheless...

This feels like dog bites man but...

SEARCH ENGINE USE HAS CONTINUED to surge in the last year, according to a new report by the Pew Internet & American Life Project and comScore Networks. Forty-one percent of 1,577 Internet users surveyed by Pew in September and October reported that they had visited a search engine the previous day. When Pew conducted a similar survey in June of 2004, just 30 percent of Web users said the same. In fact, the only Web activity more popular than searching was using e-mail; about 52 percent of U.S. Web users told Pew researchers they had sent or received e-mail on the day before being surveyed this fall.

With David's book as well. It's a combo deal. Ugh. I knew that was going to happen. But I'm pleased with it, regardless. From it:

"The Google Story" and "The Search" cast those accomplishments in opposite lights. While they overlap considerably - and while both books flatter Google with covers that mimic its brilliantly modest logo - they interpret Google's ethics and future prospects differently. The sky's the limit in "The Google Story," a glowing, sometimes credulous testament to "that Googley sense of magic." Mr. Battelle takes a more knowing, technical approach and is more inclined to look for trouble.

I'll take that.

OK, on to the FT awards. A fun day in London today. What a swell city....

I've just got back from participating in BBC radio's "Start the Week" - an analogy for which does not really exist in the US, sadly. It's a conversation between four pretty different folks, each Monday morning. Apparently a lot of people listen to the show here in the UK. Today it was me, Clive Stafford Smith, PD James, and Frederic Raphael. The program will be available here. Next up is more BBC and then CNN and some newspapers. Wow. This is all buildup to the main event tonight, which is the book awards ceremony. It's a pretty big deal, the #2 man in govt. here, Gordon Brown, will be giving out the award. I very much expect Freakonomics to win, it's the hands down runaway favorite....In any case, it's been great fun to be here, and to see a few Searchblog readers. I hope I meet many more, the ones I have met are extremely interesting folks....

A new way of bringing in endemic advertising, which is naturally site specific. This marks a new approach for Google, and an extension of its site targeting feature. I am trying to get some details on what Google believes is competitive, as this is clearly not something professional publishers with their own sales forces are going to react to well. The idea of a link "Advertise on this site" going to anyone other than the publisher is, well, controversial. But for smaller authors, it's outsourced ad sales, and they will love it. Folks like BlogAds, FM, and others will have to prove their merit even more now.

More when I have it.

Worth the time:

I suspect that Google will soon announce a program whereby people can register their "Base compliant" RSS feeds with Google base. Google will then poll these feeds regularly just like any other RSS reader. Publishers can either create brand new Base-compliant feeds or with a bit of XSLT/XML Schema of their own they can just transpose their own content into a Base compliant feed. Indeed I wouldn't be surprised if there are several software programs available for download in a couple months that do just that. Soon, every publisher on the planet will be able to have a highly automated, highly structured feed directly into Google base.

Once the feed gets inside Google the fun is just beginning. Most commentators have been underwhelmed by Google Base because they don't see the big deal of Google Base entires showing up as part of free text search. What these commentators miss, is that Google isn't gathering all this structured data just so they can regurgitate it piece-meal via unstructured queries, they are gathering all this data so that they can build the world's largest XML database.

And Mark Pincus:

google started with an amazing premise of doing no evil. i truly believe its founders want to help the world. my guess is that like many companies google will be a victim of its own success. like msft it will go hire the smartest people in the world. unfortunately, those people are often sharks and have less lofty goals, especially when they have yet to make their billions.

google base is a very msft mba approach to the world. while it makes business sense, it lacks soul. it does as little to help the community as bringing in a walmart. in fact, google feels a like walmart today.

A big brain goes to Yahoo. The release is here. From it:

Broder was a Distinguished Engineer and the CTO of the Institute for Search and Text Analysis at IBM Research. Prior to this he was vice president for research and chief scientist at the AltaVista Company, reporting directly to the CEO. He was also a senior member of the research staff at Compaq's Systems Research Center in Palo Alto. He graduated summa cum laude from Technion, the Israeli Institute of Technology, and obtained his M.Sc. and Ph.D. in computer science at Stanford University under Don Knuth.

Broder has done a lot of work in the past decade or so on search and IR, I've read many of his papers for the book, and as I recall he has done some work with folks now at Google.

Gary has some of Broder's greatest hits listed here.

Interesting article (thanks Cyril) in the Economist about the effect of search engines on traffic distribution.

...there is a widespread belief among computer, social and political scientists that search engines create a vicious circle that amplifies the dominance of established and already popular websites. Pages returned by search engines are more likely to be discovered and consequently linked to by others.

Not so, according to a controversial new paper that has recently appeared on

arXiv, an online collection of physics and related papers.

It took some searching (arXiv has a terrible search engine) but I found the referred piece here.

What I find intersting and important are the Economist's final conclusions. From the article:

The paper, which was posted on arXiv for comment, has now come under attack. Matthew Hindman, a political scientist at Arizona State University, says that the data used in the research are pretty shoddy. Moreover, he says, the discrepancy between the model and the real world does not necessarily come from the role of the search engine.

Whether Dr Fortunato's thesis stands the test of time remains to be seen. That it is tested must be a good thing.

I can't agree more. One of the things which is most frustrating about search, to me and to many, is the lack of transparency, and the lack of knowledge about how an increasingly convoluted ranking scheme actually works. Of course, Yahoo and Google can't publish their entire ranking scheme. But some kind of guideposting should be done.

This is even more true in the AdWords/Overture world, where real money is at stake, every minute of every day. I think this will come to a head sooner rather than later. For now it's all well and good to let Google determine its own profit margins by optimizing AdWords and AdSense behind the curtains of darkness. But that can't stand forever. There is too much opportunity to use that lack of transparency to ill ends - ie to bury competitors which are surfacing now by paying more than market prices to ensure that publishers stay with Google, for example. I am not suggesting that is happening, just that we have no way of knowing if it ever were to happen.

Toward that end, Seth's Root Markets is written up by ClickZ here...

In case I've not beat this drum hard enough, more thoughts on the bad law called the USA PATRIOT Act, this time from the NYT Editorial pages. From it:

Congress passed the Patriot Act hurriedly after the Sept. 11 attacks, with little time for reasoned discussion. Many of the most aggressive provisions were written to be phased out after a few years, to ensure that a future Congress would be able to reconsider them in calmer circumstances. If that were really happening, Congress would not be preparing to authorize the continued use of "national security letters," an investigative tool that gives the F.B.I. sweeping power to riffle through ordinary Americans' private records.

Unlike search warrants, national security letters do not need to be approved by a judge. The F.B.I. can issue them on its own initiative to places that hold sensitive information about American citizens, like libraries, doctors' offices, banks and Internet service providers. The Washington Post recently reported that the F.B.I. now issues more than 30,000 national security letters a year.

Update: Seems that the extensions to the Act, which were almost pushed through this past week, were shelved yesterday, though the issue will have to be dealt with next month.

The CEO of Alibaba, a central figure in China and the nascent search/ecommerce wars there, is one fun fellow to watch. His recent deal with Yahoo has redoubled Ma's presence in China. Read this Forbes piece for more. From it:

Ma isn't content to dominate China's auctions and e-mail. He wants the third point of the Internet triumvirate, too: search. The CEO depicted an almost disarmingly simple strategy: "We win eBay, buy Yahoo! and stop Google. That is for fun. Competition is for fun."

Geography and/or geopolitics apparently loom large in Ma's worldview. He told reporters that while Google (nasdaq: GOOG - news - people ) and Yahoo! dominate Europe and the U.S., neither is in a position to rule Asia.

"I call them sharks in the ocean. We are crocodiles in the Yangtze River. If we fight in the Yangtze River, we have more chances than they have."

Alas, the fabled Atlantic has recently closed, so we've moved the London get together to another location.

Floridita
100 Wardour St.
London (Soho)
020 7314 4000

See you all there at 6.30-7ish tonight! First round on me....

Covered in the Times here. Choice quote:

Mr. Adler (AAP lawyer) said Google's contention that its search program might somehow increase sales of books was speculation at best.

"When people make inquiries using Google's search engine and they come up with references to books, they are just as likely to come to this fine institution to look up those references as they are to buy them," he said, referring to the Public Library.

To which Google's Mr. Drummond replied, "Horrors."

The story is the same...

This morning (London time) I was on a panel featuring Simon Levene of Yahoo, among others. The focus was Web 2.0 and venture capital (fittingly, it was sponsored by FirstCapital). Simon, who is MD here for Yahoo in Business Development, mentioned something that struck me as both obvious yet somehow not stated very clearly. Addressing the audience of mostly VCs, he said:

"Folks like Yahoo will be competing with you for deals."

Levene mentioned to me later that Google has even set up a fund to compete with VCs for early stage company financing (I had not head this before), and that Yahoo feels it can and must compete to buy early stage companies before VCs can get in with larger financing. An interesting development. He added that entrepreneurs are weighing the risks of having to execute against the exit requirements of a second or third round of financing, vs. the bird in the hand of a deal with a big player like Yahoo, and often, as with Flickr, they are going with the platform.

UPDATE: I pinged good sources at Google, who declare definitively that Google is not running a VC fund....

Seth is starting his company based on the idea of attention markets, here is his post on the subject. I'll be reading this closely this weekend.

I penned an op ed for the Times which runs today. They've chose to run it outside of Times Select, so you can get it without paying, but registration is required. From it:

It sure feels like a bubble, doesn't it? Let's tick off the signs: a red-hot market for Internet stocks (Google, for example, has more than quadrupled since it went public in 2004); fawning articles celebrating entrepreneurs; a glut of venture capitalists elbowing one another to invest in companies with no plans on how to make money past some hand waving about "advertising" and plenty of vague claims about how their technology will "change the world."

The Internet is exciting again, and once again folks are rushing in. In some categories - like search or social networking, for example - there are scores of start-ups vying for pretty much the same market, and it's certain that, just like last time, most of them will fail.

But regardless of all this déjà vu, we are not in a bubble.

To be fair, YHOO is no slouch either.

Berkshire Hathaway, here they come....

I'm in the airport waiting for my flight to London, and since my last post on my travels, a lot of Londoners have pinged me asking to meet up while I'm there. Funny story: I was very briefly the Publisher of Wired UK, and it was my distinct regret to have to close down that office when the US company decided it was never going to make a profit. So what did I do? I invited the entire London staff to a bar off Picadilly, called the Atlantic, slapped down my corporate card, and got the whole team well and truly pissed, as they say over there.

So, in honor of my past trials and tribs, I'd like to meet any and all of you who care to meet at the Atlantic this coming Saturday. As far as I can tell it doesn't have a website, but here's some info, and a Frommer's link:

Sat Nov. 19th, 6:30 pm ....

Atlantic Bar & Grill Ltd
020 7734 4888
020 7734 5400
20 Glasshouse Street
Westminster, United Kingdom

Hope to see you all there, Michelle (my wife) and I will be there with bells on....

UPDATE: This has MOVED to Floridita, more detail here.....

Why? Because it can't respond in web time. Link.

Now there's a problem. (paid content)

It's happening: Google Base is launching. (Check the Google Blog for a post, I am sure there will be one.) I plan to talk to Google about this, though what with leaving for London and all, I may not be able to. Suffice to say, this is a major undertaking by Google, and it remains to be seen if the public will take to it. In fact, this is a test of sorts - how large will this get? Will folks trust Google with their data? Will folks see the value in it? Will Google Base become the first application that forces Google to....gasp...market its offerings? As in "List your data on Google Base, the "it" place to represent reality"?

Google is saying this is simply a new way to augment their search results. Google's right. And that alone makes it one Very Big Deal. From the PR note emailed to me just now:

Today, we launched Google Base a free online service where users can submit all types of online and offline content that Google will host and make searchable online. ....Google Base is an extension of Google’s existing content collection efforts such as our traditional web crawl system, as well as Google Sitemaps, Google Print and Google Video – all which enable content owners to easily make their information searchable via Google. The goal of Google Base is to improve the overall quality and breadth of Google Search results by collecting even more information about a wider diversity of content. ....Similar to a database, Google Base enables content owners to describe and assign attributes to it the information they upload and uses this meta-data to better target search results to what users are looking for. For example, if a chef chooses to upload their very best recipe for tamales he/she can further describe that recipe with a photo or by assigning attributes such as “medium-spicy” or “spicy.” When a user searches for the word [tamale recipes] from the Google Base homepage they will be presented with a list of recipe results accompanied by a list attributes at the top of page which enable them to further refine their search to “medium-spicy” or “hot” tamale recipes.

Rarararararghghghghghghg...this kind of disingenuous exemplar text makes me grind my teeth. This has as much to do with tamales as, well, forget what I have to say about it. Go back to the basics, and once again, read Paul Ford.

Again, from the email:

Google Base also creates a new opportunity for content producers to submit any kind of information even if it’s not a web page or online.

Behold, the physical world rendered as information. One of the main arguments in my book is that if it's not in the index, it's not considered valuable in a search-driven world. This, of course, is a new way to Get Into The Index. We've only just begun....

So I've been watching revenue estimates for the paid search marketplace lately, and I'm confused. The target for 2006 was $6 billion, set a few years ago by Piper. We're clearly past that - and a year early. But the numbers are all over the place lately. Here's a quote from the Kelsey Group, for example:

"Paid search advertising may well reach $7 billion in 2005,” said Neal Polachek, senior vice president, research and consulting, The Kelsey Group.

OK, so...Google is going to do about $6 billion in revenues this year. Toss out AOL and Ask, as they are in that gross number. But Yahoo is not. They do at least a billion and a half more in search revenues. Then there are the second tier players, who have to throw in a few more hundred million bucks here and there. And what about the Yellow Pages folks?

Anyway, with just Google and Yahoo's results to date, we're way past $7 billion. Right? Or maybe Kelsey is not counting AdSense as paid search? Whose numbers do you believe?

And you wonder why Google changed its interface to incorporate travel information? Comscore's study (jointly sponsored by Yahoo, never shy about its ambitions in the travel segment):

In April, 35 million U.S. consumers used a search engine to initiate travel planning, and those who bought travel online ultimately spent an estimated $6.6 billion in the category during the eight week analysis period.

(Hat tip: SELowdown)

Congrats to David Vise on publication of his book "The Google Story." Gary has more here.

Techcrunch has the write up.

It's the "Shoposphere." From the blog posting:

The Shoposphere and Pick Lists are examples of social commerce. We believe the community of shoppers is one of the best sources for product information and advice. The Shoposphere is a place to discover interesting and cool products thematically arranged into Pick Lists by other shoppers.

Meanwhile, the LA Times has a piece (reg required) on how the Hollywood culture is clashing with Yahoo's culture in its new Santa Monica offices.

The magazine covers how Flickr and other acquisitions has changed Yahoo. From it:

Indeed, the Flickr purchase helped ignite a larger strategy. Thanks to a new generation of managers like Butterfield and Fake, Yahoo is starting to see how user-generated content, or “social media,” is a key weapon in its war against Google (GOOG). ....

Social media “is going to be a gigantic piece of what we do,” says Yahoo CEO Terry Semel. “I don’t think old media is what people are going to spend most of their time doing on the Internet. This paradigm needs its own inventions, its own methods, its own way to go forward.”...

COO Dan Rosensweig had to overcome a lot of internal grumbling. There was no real business behind Flickr, and no unique technology either. So why did Yahoo need it? Says Rosensweig, “We could only justify it when we realized how big the vision could be if applied to the Yahoo network.”....

If Yahoo is to get social media into every nook and cranny of its business, extending it to search could have the greatest benefit of all. Targeted search ads are projected to be a $9 billion industry this year, according to S.G. Cowen. To that end, Fake took over My Web in September.....

If social search pans out, it could give Yahoo a much-needed edge over Google. Google takes an automated approach to search, throwing armies of Ph.D.s and thousands of servers at the problem. It wants to make search more relevant by creating better algorithms. Yahoo also does algorithmic search, but it can’t beat Google at that game. So it’s gambling that tapping into the collective intelligence of its audience will produce more relevant search results....

“I still think Yahoo has a heritage to overcome,” says tech book publisher and pundit Tim O’Reilly, referring to the decade-old habit of directing traffic to its site.....

The content of a suspect's Google searches were used in a murder trial. Story here, Slashdot ponders here. From that post:

Will police in the future simply serve a subpoena to Google to find out what you've been thinking about? While this use of that information makes sense, at what point does your privacy give way to public concerns? Should police be able to search through your search history for "questionable" searches before you've been arrested for a crime, and what effect would this have on the health of society?"

From the story:

Robert Petrick searched for the words "neck," "snap," "break" and "hold" on an Internet search engine before his wife died, according to prosecutors Wednesday.

Searchbloggers -

I am heading to London Weds for a few days, culminating in the FT/Goldman award dinner Monday night. I'm pretty booked throughout the time I am there, but if any of you Londeners are around and about, ping me!

Good Morning....Today:

- Google announced it is making its web analytics (Urchin, which it purchased some time ago) free to all. My big beef with Urchin is how much disk space it uses (it keeps everything so it quickly eats up your storage). So I hope this hosted solution will solve that.

Update: Not a swift rollout...

- AOL announced a big video deal (not surprisingly, with Warner). Clearly it intends to compete here.

Reg required, but here's an op ed I wrote for the SJMN today...

Late last week - and it was certainly an odd week for all sorts of reasons - I had the honor of appearing before the SDForum's Search SIG in Mountain View, on the Microsoft Valley campus. First I was interviewed by Dan Farber about the book (here's Dan's write up), then I got to interview four entrepreneurs in the search business - folks from Trulia (real estate search), Truveo (video search), Healthline (medical) and Simply Hired (jobs). Om has more on that here (though I have a rant in me about the "exit" - more later).

As usual, Dan focused his write up on what proved to be, for me anyway, the most interesting comment of the night. It came from Simply Hired CEO, Gautam Godhwani, when I asked him if he feared Google. "Google does search very well, but we have yet to see Google do applications well," was his reply.

Interesting. As I thought about that, it struck me that what we are seeing right now is indeed the evolution of search companies from their roots providing a single service - one thing, done well - to a application suite that does many things. What does that mean, exactly?

Well, in Google's case anyway, let's give credit where credit is due. Google does do a few applications pretty well. Gmail, for example, is still considered by most to be a very good mail application. Blogger, while not for pros, is also a pretty successful application ("AOL for blogs" is how one pundit put it at Web 2.0).

But neither of those are really executions of search as an application - even though Gmail has really good search. Huhm. What Godhwani was saying is that in the search field, applications are the next thing, and Google is just as new at this game as his company - if not more so in certain vertical fields.

Dan further quotes him: "Finding a job takes a few weeks or months, doing research and using the power of referrals. You can’t do it on a basic search engine, so we are complementary to search." As Godhwani said this, I was thinking to myself - "Well, as soon as there is an economic reason to do it, Google will do it, and then what?"

By then, I sense, Simply Hired (and all the other vertical search engines on the panel) hope to be so far along that the only logical move would be an acquisition, or direct competition in which the upstart actually has a chance of winning. It's how it's been for ages in this industry.

But back to this larger idea of search becoming an application. It's probably obvious to you, but for some reason this idea provides me with a way of grokking a much larger trend - why is it that Google is so focused on Toolbar, Desktop Search, Accelerator, Local, and Ajax-y things like Maps, etc.? It's because to create a decent search application, you need to have a far more robust interface, and you need to know far more about the intent of the person that is using your application. A web-based service, on the other hand, does one thing well, and does it the same for everyone. Search is becoming an application, indeed, and that more than anything else explains very well Google's recent moves.

This time with newspapers. As I said earlier when news of the magazine test started, I think this is in fact a very good thing for the industry.

It's a land grab of book-related journalism:

- Journal (free this week!) on book scanning.

- USA Today: You're missing the point!

- Larry Lessig says (Wired): This is bigger than books, bigger than just Google Print, it's about the Code! (I agree!)

Google's personal search is out of the Labs. It's based on your search history, like A9 and others.

Details at SEW here.

One thing that kind of bugs me is Google's unwillingness to call a spade a spade. From the email announcement I got:

Personalized Search also now includes several new features, such as:

• Bookmarks & Searchable Labels: Users can now create bookmarks and add searchable labels and notes to any item in their search history to help organize their information. Since these are created through a Google Account, this information is accessible from any computer by signing in to their account.

Er, isn't this what we've been calling tagging for about two years?

From time to time I get emails from the MacArthur Foundation, the folks behind the genius grants. I have no idea why, but I'm not going to question them - it's sort of nice to be asked your opinion by such a reputable place.

The last one I got asked for my input on "an issue coming over the horizon in the intermediate term where a modest investment by the MacArthur Foundation might make a substantial difference in the future.... the object of this exploration is to identify opportunities for philanthropy currently at the margin or edge...even if the optimal path of action is not certain.... focusing on those challenges where an early investment of philanthropic resources could be instrumental in mitigating negative effects or magnifying benefits for society in the future."

Here's what I came up with. If you all have modifications, input, criticisms, why, I'll pass them right along. (They asked for up to six pages, which terrified me, hence the throat clearing in the first graf....)

While I'd love to write pages on the subject, in all honesty I fear my current work schedule would only insure that I failed to respond in anything like a timely matter.

And in any case, there is one simple idea that I bring up over and over again, as the core to what I believe can affect positive change in our culture, and seems severely overlooked. That it relates to my area of expertise is certainly no coincidence.

The idea is this: we suffer - in the US, certainly, and I imagine abroad as well - from a significant lack of what I might call 21st century literacy. By this I do not mean technological literacy, though that is certainly part of it. Instead, what I find seems to be missing, and in fact, is in serious retreat at least in our public schools, is what we often call "critical thinking" - the ability to look at all the available facts and, based on reason and a sense of fairness, determine a best course of action.

Our schools are instead focused on a testing regime which requires that students focus not on solving problems or determining best courses of action, but rather regurgitating answers. But as many wiser than I have noted through the course of history, the most creative act a human can engage in is not repeating an answer, it is forming a good question.

In an age where the knowledge of mankind is increasingly at our fingertips through the services of Internet search, we must teach our children critical thinking. One can never have all the answers, but if prepared, one can always ask the right question, and from that creative act, learn to find his or her own answer.

Instead, we have leaders that believe that questions have one answer, and they already know what it is. Their mission, then, is to evangelize that answer. That, to me, is a dangerous course. Reversing it by teaching our children to learn, rather than to answer, seems to me to be a noble cause.

I then later added:

Developing a framework in our schools for  "search literacy" - how to use and think about using a search engine - might be just the kind of thing you could do with a modest investment....

I write that partially in jest, as there are many announcements today, but I have to travel again, this time on personal, family related business. Please forgive my lack of posts....and I'll post later (mid afternoon if I am lucky).

Unfortunately the personal stuff has forced rescheduling of my talk at Yahoo today (originally slated for 2 pm), and that's a real bummer. We're going to do it, but just not today. I will still be making my evening chat with Dan Farber and four search startups at the SDForum's Search SIG on the Microsoft SV Campus. Looking forward to that....

If you've been meaning to get caught up on the whole Microsoft/Web services/Web2 threat meme, then read this Cnet piece on Gates' recent memo to employees (Winer has full text there of it and Ozzie's as well), which I very much doubt was intended solely as an internal rumination, on the threats tendered by web based services. (By the way, the plasma like Cnet interface to related stories - I've used part of it as the art at left - is cool, if only we had blog plasma...)

Microsoft truly does face the second coming of the Web, and this time it's not conveniently packaged as one killable company a la Netscape (Google notwithstanding).

(image credit)

CDT, among many others, is urging all of us to take action on the PATRIOT Act, a terrible piece of legislation that I've written about before. I also featured this Act in the book. From it:

....under the PATRIOT Act, the government now
has far broader rights to intercept your private data communica-
tions—a reinterpretation of the Fourth Amendment, which states:
“The right of the people to be secure in their persons, houses, pa-
pers, and effects, against unreasonable searches and seizures, shall
not be violated.”
The PATRIOT Act certainly puts a new spin on the word
“search.” But this is to be expected, right? After all, if the government
has probable cause and a search warrant, nothing has really changed,
has it? As all good civics students know, the Fourth Amendment con-
tinues: “no warrants shall issue but upon probable cause, supported
by oath or affirmation, and particularly describing the place to be
searched, and the persons or things to be seized.”
Under PATRIOT, prior interpretations of these constitutional
presumptions don’t necessarily hold true. To summarize, the PA-
TRIOT Act holds that your private information can now be inter-
cepted and handed over to government authorities not via a search
warrant tendered to you, but rather via a request to your ISP, your
community library, or another service provider. That means that
should the government decide it wants access to your information, it
no longer needs to serve a search warrant on you; it can instead go to
the company that you use—be it Google, Yahoo, Microsoft, AOL,
or any number of others.3 In the past, the government could cer-
tainly tap your phone or search your effects if you were a suspect in
a crime. But under the PATRIOT Act, not only can the government
tap a suspect’s clickstream; the standards for who the government
can tap and how it informs a suspect have loosened as well.

Portions of the Act are under review by the House and Senate. From the CDT alert:

A "conference committee" of Congressional
leaders will begin meeting this Thursday, November 10 to reconcile the
Senate and House versions of the PATRIOT Act Reauthorization bills. At
this point, it is pretty clear that PATRIOT will be renewed, but the
Senate bill contains some important checks and balances to protect
against governmental abuse. And the Washington Post recently drew
attention to government overreaching under the PATRIOT Act. (reg required)

According to the Post, the FBI uses the PATRIOT Act to issue more than
30,000 national security letters demanding from businesses sensitive
records about their customers. The Post also reported that the FBI no
longer destroys data collected through such sweeps, even if it is
irrelevant to the investigation at hand. Instead, the article states,
the FBI has been ordered to keep the data, even when it is clear it is
on innocent Americans, and "to develop 'data mining' technology "to
probe for hidden links among the people in its growing cache of
electronic files."

More on this as it develops, first saw Jarvis's post here...Long and short: This is Google's AdWords-eBay mashup, without eBay...

Update: Internet News, Forbes coverage.

Apologies, Searchbloggers, for my absence. Travel plus some Real Life stuff have intervened on my abilities to properly post.

Meantime, much afoot. Here are some highlights.

Looksmart is branching into vertical search in a big way. Reactions are varied.

Searchblogger Chris Zaharias has started a site on SEM. Find it here.

Google launched Google Local for Mobile. Reactions here. Yahoo also launched mobile apps, integrating with SBC. More (on both) here (reg required).

Meanwhile, Yahoo hooks up with Tivo (SER).

Vint Cerf, now at Google, writes at length on the issue of net neutrality, which is heating up due to hearings on the issue, and I'm guessing that there is a serious chance the Bells and their ilk may well win this.

Just so you guys know, I did read the NYT on Yahoo and Google this weekend. Honestly, fine pieces. I just was traveling Sunday....

Murdoch sees opp in distribution business in US. Target: Comcast.

Topix stripes its service via 15,000 weblogs. Details here.

Wired sponsoring a debate on Google Print. This should be good.

MSN AdCenter is improving (SEW). Met the fellow behind this at Adtech, looking forward to learning more.

I'm at ad:tech Monday, in NYC. Back online Tuesday...and this Thursday, I'll be part of the SD Forum's Search SIG. From the description on Jeff Clavier's blog:

The topic of this month is “The Search: A ten year perspective”, during which we will look back at the early days of the search industry, its key turning points, and discuss its short term outlook and its future.
The event will be held on the Microsoft Silicon Valley Campus, Mountain View. Hope to see you there!

First we had readers in the Dublin airport, now it's the Munich airport, with reader Oliver Thylmann deep in the pure literary joy that is The Search (nice bubble chair, buddy).

By the way, last week you guys pushed the book to #10 on the BusinessWeek best seller list. THANK YOU!

Kevin Kelly sent this my way, and it looked like a hoax, but I don't think it is, nor does Kevin, and the b'sphere is all over it. Amazon Mechanical Turk. From the overview:

In 1769, Hungarian nobleman Wolfgang von Kempelen astonished Europe by building a mechanical chess-playing automaton that defeated nearly every opponent it faced. A life-sized wooden mannequin, adorned with a fur-trimmed robe and a turban, Kempelen’s "Turk" was seated behind a cabinet and toured Europe confounding such brilliant challengers as Benjamin Franklin and Napoleon Bonaparte. To persuade skeptical audiences, Kempelen would slide open the cabinet’s doors to reveal the intricate set of gears, cogs and springs that powered his invention. He convinced them that he had built a machine that made decisions using artificial intelligence. What they did not know was the secret behind the Mechanical Turk: a chess master cleverly concealed inside.

Today, we build complex software applications based on the things computers do well, such as storing and retrieving large amounts of information or rapidly performing calculations. However, humans still significantly outperform the most powerful computers at completing such simple tasks as identifying objects in photographs—something children can do even before they learn to speak.

When we think of interfaces between human beings and computers, we usually assume that the human being is the one requesting that a task be completed, and the computer is completing the task and providing the results. What if this process were reversed and a computer program could ask a human being to perform a task and return the results? What if it could coordinate many human beings to perform a task?

Amazon Mechanical Turk provides a web services API for computers to integrate "artificial, artificial intelligence" directly into their processing by making requests of humans. Developers use the Amazon Mechanical Turk web services API to submit tasks to the Amazon Mechanical Turk web site, approve completed tasks, and incorporate the answers into their software applications. To the application, the transaction looks very much like any remote procedure call: the application sends the request, and the service returns the results. In reality, a network of humans fuels this artificial, artificial intelligence by coming to the web site, searching for and completing tasks, and receiving payment for their work.

There's much AdSense news today, including a new affiliate program that lets AdSense publishers make money by referring folks into AdSense, and - surprise surprise - make a buck (literally) for everyone who might download and install Google's Toolbar.

But what I find fascinating are the new AdSense Terms of Service, which I am just digging into (thanks Glenn). The industry seems focused on the new referral stuff, but it seems they have changed to redefine what Google finds competitive to their network - now that the company has image ads and site specific stuff, they need to start boxing out new kinds of competitors. Anyone out there fully grokked this? I plan to later this weekend....

I've been watching the number of posts on Searchblog gradually head up to 2000, but not till I saw that the last one was # 2004 - and this # 2005 - did I realize the milestone had been reached. And, Searchblog turned two years old last month, another milestone I forgot to note! Here is number 1, and here's number 1000....

Andrew Goodman points to this patent, which is filed by folks who work at Google (though Google is nowhere in the filing...hmmmm.) From Andrew's post:

Google sees its algorithmic thinking increasingly as applying to all "placed content." This can mean organic search results, ads near organic search results, ads or related headlines near email, or ads on content pages.

Personalization potentially creeps into the way that ads are displayed, then. That'll eventually have a dramatic impact on the opportunities available to advertisers, and the price they may pay to gain visibility.

From the Patent:

What is claimed is:

... A method of personalizing placed content, comprising: determining an interest of a user; accessing a user profile associated with the user; identifying a set of placed content that matches the interest of the user; and ordering the set of placed content in accordance with the user profile.

... A method of personalizing placed content associated with a search query, comprising: receiving a search query from a user; accessing a user profile associated with the user; identifying a set of placed content that matches the search query; and ordering the set of placed content in accordance with the user profile.

... A method of personalizing placed content associated with a search query, comprising: receiving a search query from a user; accessing a user profile associated with the user; identifying a set of placed content that matches the search query; assigning a score to each of the set of placed content in accordance with the user profile, a respective bid value for the placed content, and a respective click through rate for the placed content; and ranking the set of placed content according to their scores.

It goes on and on...the implications are rather far reaching, though not surprising to readers of this site....This is juicy reading....

The Wall St. Journal - which loves this kind of stuff - has a "their's is bigger than yours" piece (reg required) about the pair's new 767 jet - which they bought for personal use.

On the road, Sergey Brin and Larry Page have owned environmentally friendly hybrid vehicles such as the Toyota Prius. In the air, they apparently prefer something roomier.

Google Inc.'s two billionaire founders, both 32 years old, will soon be cruising the skies in a Boeing 767 wide-body airliner. They bought the used plane earlier this year, Mr. Page says....

Tech moguls delight in public one-upmanship and the Google founders' 767 raises the bar. Microsoft Corp. co-founder Paul Allen owns a fleet of aircraft, but his flagships -- two Boeing 757s -- are smaller than Messrs. Brin and Page's 767. It also marks a new level of consumption by the Google executives, who have shunned most trappings of the super-rich despite a combined net worth estimated at more than $20 billion.

The piece is fascinating for its obsession with detail about this purchase.

Here's fun with photos: Meetup's Scott Heiferman was passing by the HQ of Honeybee Robotics - the folks who are helping NASA build the Mars Rover - and he noticed this welcome message (full photo here):

A passing curiosity, I am sure.

Jeff Bezos made a lot of money from his investment in Google (he was an early investor), and Amazon's A9 builds on Google's search service, but today Amazon announced it is "introducing two new programs that allow consumers to buy online access to portions of a book or to the entire book, giving publishers and authors another way to generate revenue from their content" (quote from the Journal piece, which is behind a paid wall).

Another tidbit from the story:

While Amazon Chief Executive Jeff Bezos wouldn't comment specifically on the Google Print controversy, he said, "It's really important to do this cooperatively with the copyright holders, with the publishing community, with the authors. We're going to keep working in that cooperative vein."

Not to toot my pal's horn, but if you want to see this model really working well, check out O'Reilly's Safari service.

Meanwhile, Boing Boing has more coverage of the Google Print story, this one a rebuttal to the AAP's Pat Schroeder.

Yahoo late last night announced a new set of APIs for its mapping application, and a new beta of its Maps application. Yahoo is aggressively looking to expand its participation in the mashup world with these new hooks. Notably, you can hook into local search as well...and, presumably, the biz model for same as well.

Update: Thanks to SEW, here's a good overview of Yahoo's new offering. I'm heads down on a few things today...

Today announcing scads of out of copyright books are now available for full reading....

A whileback I asked you all to take a survey so I - and my new company FM - could get to know you better. The results are in, and I wanted to share them with you. Here they are. If you are like the typical Searchblog reader, you are in your early 30s, male, live in the US (on the West or East Coast), make more than 100K (wow!), have a really good job, don't have any kids, and use the web a hell of a lot.

Those of you who've read my book will recall that the whole story started when someone sent me a link to Google's original Zeitgeist - the top gaining and declining queries for the year 2001. Since then, ol' ZG has been hitting the lists every week, and I check it regularly. And I have to say, it's beyond predictable - now it's simply deadly dull. Here's this week's top five gainers:

1. rosa parks
2. halloween
3. daylight savings
4. jack-o-lantern
5. costume ideas

Come on Google, you know so much more in your own version of the Database of Intentions than you are letting on. Think what you might be able to surface with a bit of imagination? How about some of you Googlers start futzing around in that fabled 20% time, and come up with something interesting again?

Googler Matt Cutts, whose site I recommend if you want to stay current on that which Google allows him to talk about (usually Adwords) has an interesting post on the practice of hard coding a response for a particular query, in this case, "Google."

Loren points out Yahoo has hard-coded some special behavior for the query [google]. Searching for [Google] at the Australia/NZ version of Yahoo returns “Try the new Yahoo! Search:” with a prominent search box right above the listing for Google. Interesting..
This reminds me of similar incidents from other search engines in the past. Remember when Inktomi hand-coded the result for [dumb motherfucker] so that the #1 result was a Google page about its executives? Boy, I do. That was motivating.
But let's be honest here. Everyone does this. Google might do it through targeted AdWords, but it's hard coding by another name...

Some choice words for Microsoft from Russell (who you may recall now works at Yahoo).

Though it doesn’t seem to make sense for Live to have the Windows or Office names right now - live.com is just another web dashboard at the moment - there are far more ambitious plans to come. The Windows and Office monikers are there because Microsoft will, of course, be up to it’s old tricks by heavily integrating Live services into the desktop sucking the air supply out of any online competitors. It doesn’t seem that they should be allowed to do this sort of thing, but the success of iTunes seems to have given them a new excuse to start tying products again. And hell, the DOJ agreement only lasts until 2007, no? ....Yep, for Microsoft, Live isn’t about Web 2.0 at all, it’s all about Monopoly 4.0.

Update: Om adds a graphic here.

Slashdot has the chew through, but I think this is utter speculation. Of course, I thought Gmail was a hoax. The original article which spawned this rumor is here, that piece*is* utter speculation.

Earnings at IAC were boosted by its Ask acquisition, Rafat reports.

Gates is unveiling an online version of Office, and "Windows Live" - a MSFT answer to the rest of the web. More as it comes in.

Here's more from MarketWatch. Nothing surprising ... but it's on.

And Cnet's coverage:

Kicking off what he called the "live era" of software, Microsoft chairman Bill Gates on Tuesday said the company plans to launch new Internet-based complements to its core products.

Gates said Microsoft is working on two new products: "Windows Live" and "Office Live" that create new opportunities for the company to sell online subscriptions and advertising. Both are targeted at smaller businesses and consumers.

The new products won't replace the company's ubiquitous operating system or productivity suite. "They are not required to use Windows or Office," Gates said.

Google's back at the scanner (WSJ free link), despite the lawsuits. The company had planned to start again Nov. 1, after halting its program to see if negotiations might bear fruite. They ain't.

I've always enjoyed talking with Perry Evans, CEO of Local Matters, and found that his company is a quiet but interesting contender in the local search space. Why is that? Well, the partnerships with Dex (a Yellow Pages company) among others. And the fact that Evans is the guy behind MapQuest, so he knows his way around this territory. Today Local Matters announced new funding and an acquisition, of AreaGuides.net.

Local coverage (Denver Post) here.
Release here.