John Battelle's Searchblog

Ya gotta love the Yellow Pages - all printy fresh. In a press release issued today (the link is an instant download and I imagine you don't need the pdf clutter), they offered up their top searches for the year - the year 2002. In any case, it's interesting to see what the top "references" are - their terminology, best as I can tell, for the equivalent of a person taking action based on a Yellow Pages listing. This data comes from the Yellow Pages Integrated Media Association, which exists mainly to promulgate the idea that the Yellow Pages are a vibrant and long-lived source of leads for local business. Caveats herewith in place, the top ten are:

1. Restaurants
2. Physicians & Surgeons
3. Automobile Parts-New & Used
4. Automobile Repairing & Service
5. Pizza
6. Automobile Dealers-New & Used
7. Beauty Salons
8. Attorneys/lawyers
9. Dentists
10. Hospitals.

The data also includes the actual number of references for each term. The top term (Restaurants) had more than 1.3 billion references. In other words, folks used that category in the Yellow Pages 1.3 billion times in 2002. How on earth did they came up with this number?The study of course has no methodology attached to it. I mean, how *do* you track this? Compared to paid search, which is entirely trackable? Anyone know?! Guess I'll have to call the YPIMA and ask. Also, I've emailed folks at Google and Overture to ask what the equivalents are in paid search, which would be a fun comparison. ...

UPDATE: Click on the (more) link below to see the full response from the Yellow Pages Integrated Marketing Assocaition on how they got this data...Thanks to the IMA for this response!

Continue reading "(Updated x2) Top Yellow Pages Searches" »

A Nielsen/NetRatings report shows how widespread the use of web-enabled applications (ie Chat, iTunes, etc) has become. MediaPost reports.

Back in 1997, Wired ran a cover story called "101 Ways to Save Apple." The cover remains my favorite for a number of reasons, the brilliance of the image, the genius of the singular imperative "Pray." I'm not sure the story, in which we polled a bunch of folks and created a list - Editors LOVE lists - was that great (actually, point #101 was pretty good: "Don't worry. You'll survive . It's Netscape we should really worry about.")

This month Wired is revising the 101 Ways meme with 101 Ways to Save the Internet. The story was written by Paul Boutin with input from a few key folks, including several readers of this blog. The voice is almost right, the politics line up, the issues are well chosen, but something about the list feels a bit off. I can't put my finger on it, but overall, it's a good rundown of all the issues the Net faces as we head into 2004.

OK, here's a new idea: Search for people based on their business cards. I kid you not. CardBrowser is a web-based, paid registration database of...business cards gathered at various high tech conferences (more than 100 a year, they claim).

Now, nowhere on the site can I find exactly *how* they gather those cards, or if the folks represented on those cards are aware they are in a database, but...I've called to find out and will report back when I do.

The company behind CardBrowser is marketing the database as a way for companies to find "passive" job seekers - folks who already have good jobs in high tech who might not be actively raising their hands for new jobs. Recruiters can buy a subscription to the site and then contact potential recruits - and, the site boasts,have a pretty good chance of getting a response, as the information on a business card tends to be accurate.

This brings up a rather odd catch 22. Now, if the folks who are in this card database - and the company claims to have more than 17000 names, with some 2000-4000 added each month - *do* know they are being added to this database, then well, they ain't exactly passive anymore, are they? As an employer, I'd be less than happy to discover some of my key people in this database, and were I the distrustful type, I'd probably get a subscription just to check. If, on the other hand, the folks do *not* know they are in the database, seems to me we've got something of a privacy problem on our hands. The company has no privacy policy I could find, and does not address this issue anywhere. Could this be a simple oversight?

Tara over at ResearchBuzz has found this new entry: Ay-Up. Yup, Ay-Up. As she points out in her post summarizing its features, Ay-Up is unique in that it offers free site search. Worth a looksee. (And yet another new engine that needs the help of a logo specialist...)

The latest entry in Odd Little Search Engines That Might...Sootle. Please, let me know if you want me to stop pointing you to this stuff. This engine is in deep Alpha, which might explain its name, logo, and terrible results (30 results for George Bush...) but not the lack of grammatical coherence in its "about" section...Given that the name of the Financial Director is "Peter Fiasco", I'm beginning to wonder if these new sites aren't elaborate jokes tossed up late at night by overworked engineers at Yahoo or Google....I mean....Sootle?

UPDATE: Within 12 hours of my posting this, both Peter Fiasco (my apologies, he's apparently a real guy) and the founder of Sootle, Sid Yadav, emailed me. They were quite kind, pointing out that my criticism of the site would inspire them to greater things with their new creation. Sid points out that my Bush search in fact found 30 *clusters*, and a total of 313 results. His index is only 11 days old, and is only starting to crawl ...literally. He calls Sootle "a hobby sort of project" and is working on a new logo and interface. Stay tuned....

Folks from time to time ask me if the game is up, if Google, Yahoo and a few others have locked up search forever. Bah!, I say. Search lives in every corner of the internet, insinuating itself into every fold of discoverable information. There are simply too many folds - large search companies can't profitably exploit every one of them. Hence the continued rise, in 2004, of the vertical search category. Examples? Sidestep, a travel search site that seems to be gaining traction of late. Why? The company's core promise to consumers can be found on its site: "SideStep is a search engine - not an online travel agency. " In other words, you can trust it, as it's not trying to sell you anything. It's focused entirely on its mission of finding the best travel deal, as opposed to selling you whatever inventory its partners might want to clear that day.

So what's the point? Have I fallen in love with Sidestep? No - it's still fish with feet - requiring you to download a software application that "watches" you do travel searches, then makes better suggestions. But I just love the idea it represents: search is a real time publishing opportunity. You can make a business of solving a person's ephemeral but specific information problem, addressing a person's simple but non-trivial query - "What's the cheapest hotel room in New York right now?" - and make a decent living at it to boot. Obritz, Expedia, Hotels.com - they all claim to do that - but they're not publishers, they're agents. Same with so many other first-generation vertical sites - Autobytel comes to mind. My experience is that they are all in the thrall of their partners and their inventory - they are in no way independent. (Just try asking Autobytel this question: "What's the cheapest Volvo c70 on the Web right now?" They send you to a dealer. Not exactly what you had in mind, eh?) I just love the idea that finding an honest answer to a reasonable question works as a business on the internet. Somehow, it feels like the essence of what publishing on the web can be - impartial (and complete) answers to honest questions. So I root for the Sidesteps of the world. The idea it represents scales to all sorts of opportunities, yet to be discovered.

The FBI recently issued an alert for all police nationwide to be on the watch for folks carrying almanacs, in particular those that might be "annotated in suspicious ways." Now, I just can't imagine they didn't realize how profoundly stupid this would sound to your average citizen. I mean, did they? The AP story goes on to quote the text of the FBI alert: "The practice of researching potential targets is consistent with known methods of al-Qaida and other terrorist organizations that seek to maximize the likelihood of operational success through careful planning.'' So if this is a serious "alert" - and as far as I can tell, it's not April 1st - one can only imagine the red flag the FBI will wave once they figure out what kind of "research" Google can do. Given the timeline they seem to be on, that will be in 2103 or so.

Yet another pretender to the throne, Mooter.com is a new search engine out of Australia. It uses clustering technology - not a new idea - but claims to have made it better. I tried it (quite cursorily) and it was, well, not awful. Scattered reports say the site sometimes fails to return results, but that hasn't stopped the Mooter CEO from saying they plan to go public on Google's coattails. At least she's being honest. According to a local news story on the IPO, Mooter has no profits. And if you hit their site and poke around, they sound darn flaky. From their Mission Statement:

As we move, as we track through the information now presented, as our brains cavort along their apparently random paths, increasingly powerful technologies will anticipate our needs. ....We must keep thinking. About who we really are. About what we really want. We must have a powerful tool for finding our way around the information world: a tool that does not impose value on us, but helps us find our own meaning.
If we do not, the mutated survivors will be the corporations who have managed the most manipulation, not the beauty of the human spirit in all its fierce joy of living and intensity of love for self and other sentients.
We must be mindful of what we plant, our children will bear the fruits.

Dude, pass the bong. It's sophomore year again.

And if they do go big time, they gotta do something about the logo. It looks like the mascot for a cold medication.

There's been a lot of noise over Comcast lately, most of it about the company's rather restrictive terms of service for their broadband product. (It boils down to this: They make it hard to do anything but take a high bandwidth feed from them. Thus they are approaching the internet, predictably, the way they approach cable - a dumb system with intelligence, such that it is, embedded in the servers, rather than at the nodes). So watch this space: Comcast is continuing to flex its programming muscle. This means that Comcast the ISP will act more and more in the interests of Comcast the owner of entertainment programming. Which net net, isn't going to be good for the net, at least in the short term.

This is a very big deal. Not just because of the law itself - it's heinous - but because of the way it was passed by the Bush administration - on a Saturday, during Saddam's capture celebration, after an unaccountable voice vote on Thanksgiving with no debate. And of course the media did not pay attention, and of course, I hope, we will. I'll summarize the effect of this later - it gives the government extraordinary new powers of search (which is one reason it relates to this site) - but for now, please give your lawmakers hell.

If you're a linguistic geek, or just like stoning out on how words work, check out WordNet. I was told of this site in an email discussion with a reader, it's an ongoing academic research project based out of Princeton.

This site (it's also available as downloadable software) is basically a database of interconnected word meanings. The site says: "WordNet® is an online lexical reference system whose design is inspired by current psycholinguistic theories of human lexical memory ." Er, in other words, it's a neat way to see the various "senses" a particular word might have. The online site has a Word Search function. Type in any word, say...."search"...and you'll see it has five senses as a noun, and four as a verb. You can then explore various aspects of the word's senses, including synonyms, derivations, and - really cool - hypernyms: "Search is a kind of..." and hyponyms "..is a kind of search."

I'll admit, this was my first time stumbling across the terms hypernym and hyponym and actually understanding why they matter. So why does this matter to Search writ large? Because one way to think about improving Search is for an engine to drill down on a particular query based on what sense of the word the searcher intended. In other words, when you type "jaguar" into the query box, which sense did you mean - the cat, the car, the team, the software? If a search engine can create "senses" of words on the fly, it might be able to create smart responses to difficult and high-results queries (AltaVista and others do something similar with clustering, but this technology has not been blessed by everyone as relevant enough..including Google). Think of Google's spell checker, but with "senses" of words, instead of spellings of words. "Did you mean the cat?...." etc. Now, I have no idea if this particular implementation would be useful to a search engine, it probably has all sorts of problems. But it's interesting to think about nevertheless.

Thanks to Steve K for this pointer and the conversation that provoked it.

(An aside - my email is jbat@battellemedia.com. I welcome email if a particular thought of yours is more comfortable in that medium as opposed to the site's comments area. I've learned a lot from such exchanges).

In the English language version of People' Daily (take it for what it's worth...) is a rather exuberant announcement for the launch of the "world's largest Chinese search engine", known officially in English as "China Search Online" (www.zhongsou.com). The page is reasonably clutter-free, as compared to most Chinese portals I've seen (I co-taught a course on weblogs and China last semester, the product of that course is a cool weblog called China Digital News.)
In any case, the folks behind the engine, HII (who went public earlier this month, see here) are compared to Google, they even have a no-human-editors-have-touched-this news product to boot.

In an email conversation, Danny Sullivan (he of Search Engine Watch fame) and I recently were discussing last week's post on Froogle. Danny disagreed with my premise that Google's actions were inconsistent, in fact, he believes they may well be consistent with a new and evolving interface approach that he calls "invisible tabs." He explains the idea here. The gist: search engines will intuit what you are looking for behind the scenes, and deliver to you the results most consistent with that intuition, making the tab format redundant in the first place.

As Danny put it in an email to me:

The real departure is going to be if Google finally makes the jump and gives you back 10 product/Froogle results at some point, and suggest that you might also search the web, for some queries, rather than the web dominance we get now. That will be them fully putting into play this whole invisible tabs concept that I've been talking about recently.

Danny points out that Google already does this with News. Try searching for "George Bush," for example. You'll see News results at the top. Google is intuiting that you wanted news on George Bush, or at the very least, that news about George Bush is relevant to your search.

Same thing for Froogle results, Danny explains: "They're hitting the Froogle database in automated fashion, and if the automated system feels confident enough, you get Froogle results displayed. No different really in look, feel and operation than searching for "iraq" and getting news results."

Well, yes...and no. What I find interesting is this part of the idea: " If the automated system feels confident enough, you get Froogle results displayed." No matter what, code = architecture, and architecture = politics. Somebody had to code that Froogle algorithm to determine its confidence/intuition with regard to your search. Google, and any other search engine worth its shareholder's money, will never tell you how it makes those decisions. They are the Invisible Hands of the automated search process. The men behind the curtains.

And therein lies the interesting bits.

Regardless, we should all give Google a lot of credit for having neither paid inclusion nor referral fees in their shopping engine. That is leaving a lot of money on the table toward a greater end, and an indication of the philosophy which guides them.

More and more I'm noticing my cable lineup looks like a magazine rack. Used to be, television was a scarce resource. As late as five years ago, it was still being programmed for large audiences - at least a million, if not more. If folks wanted well-produced niche content, they had to go to magazines. Now they can go to the internet as well, but until recently, I thought magazines could still compete for a smaller audience's attention if they stood out as a voice for a particular community.

But I now believe magazines as we understand them are eroding, succumbing to the twin tides of niche cable and what might be called the second wave of Internet publishing.

First, TV. Cable seems to have finally realized that in a 500-channel universe, not every channel can garner a 20 rating. Hence a willingness to do focused, niche content that aspires to just several hundred thousand viewers at a time. This strategy can produce breakout mini-hits like Trading Spaces and Queer Eye, but in general, it seems cable has figured out how to make money selling audience sizes based on metrics quite similar to those of magazines. Thumbing up and down my cable menu, I feel like I'm at the magazine rack at Barnes & Noble - there's 25 different sports titles, scores of shelter books (that's the home/hearth category for you non-magazine folk out there), plenty of music/pop culture plays, even programmatic equivalents of "Guns&Ammo." None of these shows, save perhaps the pop culture stuff, do more than 500K in audience on any given day. In other words, TV has managed to segment audiences into the same demographic/psychographic buckets that once were the sole purchase of magazine land. PVRs only accelerate this trend, adding the convenience of search and storage to the magazine rack concept. Add in the fact that the average cable bill in the US is more than $40, and you have a subscription+ad model, just like magazines. I should also note that the advertising business has shifted in kind: production costs have been driven down by technology, and buyers now understand how to buy spot and niche cable. End game: TV wins head to head against print. Just ask the publishers of Life.

Now, the Internet. I've always thought you could create great magazines if you stayed away from competing solely on audience demographics/psychographics, and focused on the ineffable quality of publishing that might be called community. Because they serve deep and subtle content, magazines can create and/or declare community, a badge that folks wear proudly, a club in which they claim membership. Well, while it's quite difficult for a cable show to hold this rather ephemeral quality, the Internet has it in spades. Strike two.

If my beloved magazine readers are getting their high-bandwidth niche experience from cable, and their community succor from the web, what is left for magazines to do? Is Battelle saying magazines are dead?

Well, yes and no. We have to rethink what a magazine is. Again. After all, I teach magazine development, for goodness sake, so I can't very well believe magazines are a dying breed. There are exciting things to be done with the idea of magazines, if we can reinterpret them. For one, make them for smaller audiences, and compete on that point, rather than be ashamed of it. Two, figure out how to make magazines sing in the online world (nearly all attempts to date are awful). And three, figure out a way to get around traditional approaches to the twin evils of circulation marketing and distribution. I'll post more on my thoughts as to how at a later date, but I wanted to get that cable TV-as-magazine-rack meme out there, and see what you all thought.

As long as I'm skipping down memory lane and all, I wanted to leave one tech/site related request - late though it is - for Santa this Christmas Eve. Or maybe I should say, for Ben and Mena over at Six Apart (the folks who brought us the software that makes this site, and many others, possible). I wish for Quark Xpress for the Web. I know this has been something of a pipe dream for many (and I'm not really tight into the discussion on this topic, so perhaps it's on its way), but now that I've mastered a few of the basics of Moveable Type, I really wish I could play with the software the way I did when desktop publishing was young, back in the late 1980s, when Quark and others made history. One of my first gigs was as a paginator for MacWeek, which claimed to be the first desktop-published four-color national weekly magazine in history. It was just really really cool to be a beta tester of software (Quark version .9) that you knew was going to change publishing forever. As great as MT is, it's still too hard to tweak the sites, to make them look better and perform better from a reader's point of view. I know making this stuff is extra hard. But I'd sure love to beta test the GUI version of MT, were it ever to come down the chimney....

And Happy Holidays to you all. Thanks for reading these past couple of months, and giving me so much to chew on. I look forward to 2004.

Because it's poised to grow to nearly $3 billion in revenues by 2008, up from about $1 billion now. And because the mass of small business owners who currently don't use search would certainly switch if presented a compelling solution that actually brings in customers. Can you imagine your corner grocery store or dry cleaner buying keyword search? Me too. Move over, Yellow Pages....

Before there was the web, there was the corporate database. Remember those days? Back in the mid to late 80s, when the Local Area Network was the Next Big Thing, when everyone was madly installing client-server databases, when applications like dBase III and NetWare ruled the roost? You don't? Sigh. I must be showing my age. I was a cub reporter back then, covering the relatively new beat of "networking" as well as the corporate database market. Yup. Somehow I found that stuff fascinating. I thought this whole idea of connecting disparate networks of information was a hoot.

Anyway, to the point. About 1987 or so a new class of applications developed. Called Executive Information Systems (EIS), these were essentially interfaces to data, designed to live on top of corporate databases and cull the stuff Really Important Executives needed to know so as to make Really Important Decisions. The coolest part of the spec was the fact that the data was queried from the desktop - EIS promised easy and intuitive access those unintelligible databases the geeks kept buying. The idea was sexy, but the category never really took off. The design was too rules based, too top down. For them to work, you had to literally redesign your entire infrastructure. Oh, and the Executives in question had to give a shit.

Fast forward to now. As most of the world remains fascinated with search's more public face, a significant shift seems to be occurring in the corporate data world. I'm not saying EIS is back, exactly, but the overwhelming presumption of webwide search on your desktop is certainly rewiring how corporations think about their more private databanks. A robust market has grown up around "enterprise search," (some companies, such as FAST, were spun off from consumer search companies, and Google maintains a unit focused on the market). There's a crop of interesting startups to boot, including Tim Bray's company, Antarctica. It's entirely possible some of the next big ideas in search may well be developed in this more focused, less public field. Any readers out there have suggestions of cool companies in this space I may be overlooking?

Wired News reports on a massive database project from Mayo which has interesting, scary, and rather exciting implications for diagnosis and treatment. Genetic information will eventually be included. Excerpts:

During an office visit, a medic will be able to do enough quick data mining to ensure the most accurate diagnoses and most effective treatments while the patient waits, de Groen said. "Ideally the computer would query both our own database of patients (and) the complete medical literature." ....

Health-care professionals look forward to the eventual addition of patients' genetic information to databases like the MCLSS -- a field known as clinical genomics -- as a major advance in medicine. Among other things, such access would allow doctors to divine with great speed and accuracy what drugs have worked best on a certain type of person with a certain illness. ...

"It's really about applying the knowledge from many to the benefit of one," said Dr. Anne-Marie Derouault, director of alliances and distribution channel management for IBM Life Sciences, who heads the IBM teams working on the project. "Without genomics, it would be very hard to do that. Putting that kind of information with traditional information is potentially going to bring medicine to another level."

I am not sure why all of a sudden I am struck with the urge to prognosticate, but all weekend long I've been thinking about what might happen next year in the search/tech/media nexus. I think it has something to do with the book - my plan is to finish it by about mid year, then pray that nothing major changes for another six months while the manuscript wends its way through the vagaries of the publishing process. It's either that, or Jeremy envy.

So I've been thinking about a number of things, some small, some not so small, which might happen in the next twelve months. Given that I'm writing this on the eve of Winter's Solstice, I give you Battelle's First Annual Solstice Hopes and Predictions for 2004. I refuse to say which are hopes, and which predictions. This way, I can claim to be right next year one way or another. Take it for what it cost you on the way in.... (see list via link below)

Continue reading "Thoughts on 2004" »

I subscribe to Bill Gurley's Above the Crowd newsletter, and always find his insights worth the read. (Bill is a VC at Benchmark, and a former partner of mine on the Internet Summit conferences.) His latest missive, "Cleaning Up After the Ninth Circuit in an Attempt to Save the Internet" is an exercise in counter-intuitive thinking. When I first read about this decision, I thought it was a victory against the evil cable companies, who I am always willing to believe have nothing but their own monopolies at heart. The ruling said, in short, that cable lines had to be viewed as telecommunication services rather than information services. The distinction was important, as it meant the cable lines were subject to the same sharing rules as phone lines. In other words, it could open cable up to competition, similar to what the RBOCs already face.

Given that for three of the past four years SBC felt my neighborhood was not profitable enough to offer DSL, and I had to get it from Speakeasy, which offers service on top of SBC lines, I thought this kind of a ruling was a good idea - maybe there would finally be someone like Speakeasy who could offer me cable modem speeds and who had a clue about the internet (Comcast clearly does not - don't get me started on that one).

Bill makes the case that while the intent of the court may have been good, the result could be crippling. I am not sure I agree with everything he writes - much of it is pretty rigid anti-regulation sentiment - but he makes some good points. Among them:

...Often in complex political systems, the objective of an action can be honorable, yet the impact of said action can be completely at odds with the objective. This is largely because the tools we use to encourage behavior in such systems are often crude and imprecise.

Attempting to increase competition by mandating that a company invest in infrastructure and then share that infrastructure with competitors is simply not a market-based solution. ...

We should all know by now that rather than increasing competition, regulation typically reinforces monopolies and oligopolies. Startups will not and cannot prevail in heavily regulated industries. ...

Bill concludes that either the Congress of the Supreme Court will have to clean up the Ninth's mess.

(Updated)
I think it'd be cool if you could join a network of folks who read the same blog(s). I've always maintained that any good publishing effort understands and reflects its community - that it is both a mirror to the community members, and a window into that community for folks who are interested in joining or understanding that community. Conferences have always been a neat way for readers of a publication to meet each other, for example. Foo Camp was one of the first I've been to where "blog ecologies" ended up meeting FTF, and it was quite something to see how folks who'd been connected mainly by blogs ended up working together in real space.

So think if you could "see" all the other people who read this site each day (and who opt-in to be seen, of course) - and invite them into a LinkedIn like network if you wished to. I wonder if that's in the cards for LinkedIn - to do vertical OEM stuff like that? Are there others working on stuff like this?
(Thanks for the meme, Matt...)

I keep meaning to do this...I've noticed a bunch of net-based aggregators out there, I'd be curious if anyone uses them. One of the earliest is Bloglines, founded by the same fellow behind eGroups. There's also a new one over at Feedster call myFeedster. And there's FastBuzz. And of course the one Meg's working on, Kinja (still in stealth).

So Marc's company will go first - the first of the companies on everyone's list of '04 IPOs to drop an S-1 at the SEC's doorstep. Here's the Merc's story on it...

(caveat: site related)

A few readers of this site may recall the naming discussion we had back in 1997-8 when The Standard was born. Back then I was trying to come up with a name that properly captured the intent of the magazine, and I had several dogs in contention before Denise Caruso, Steven Johnson and I came up with The Industry Standard over (more than one round of) drinks in NYC. During that exercise, and during many subsequent and prior naming go-rounds, I always started email lists with trusted advisors, pinging them with naming ideas and getting their feedback.
This is a long way of saying that one of the reasons you see the "beta" tag on the banner of this site is that I've always been dissatisfied with its name. Somehow attaching the word "blog" felt dated, but I did it anyway, if only to signal the form the site would take. So with that in mind, I seek your counsel. You've been reading for a while now. Which of these names capture best the feel of this place? (I'm partial to the idea of keeping Search in the name, as you can tell).

-SearchThink
-SearchThought
-Searchlog
-SearchQuest

Have any others? If I rename the site to your suggestion, you get bragging rights forever, just like Steven and Denise have. Think of the whuffie....

And as long as I'm doing site news, I've now updated my templates to support RSS 2.0, so you can see links and pics in your readers, if they support 2.0.

This meme has popped up a lot over the past decade: We're increasingly putting everything on digital media, which is great, but we're failing to archive it properly, and even when we do, we're not archiving the machines we need to read the data. A librarian at the British Library makes this observation again in an article entitled The Great Digital Information Disappearing Act. (Thanks Gary for the link).

But we're missing a larger point. It's not just the data - the emails, MP3s, the websites. It's also the data about the data - the traces of our digital culture that are kept in the database of intentions. That data - what people wanted, when they wanted it, how they asked for it, what they got, where they went - represents no less than the cultural artifacts of our day, equivalent to the pottery shards and stone tools left by our forbearers. Imagine yourself an archaeologist 2,000 years from now. Wouldn't you like access to MSN or Google's database of intentions, so as to plumb the traces of an ancient culture emerging into the digital era? But to date, most search companies are either not archiving that data, or if they are, they are keeping it to themselves for competitive or legal reasons. There are huge privacy issues, of course, in insuring this data becomes part of our digital history, but they are solvable. What we don't have is the cultural will or foresight to realize what we're creating.

In any case, this is one of the larger informing concepts I am working on for the book. Your comments and input greatly appreciated.

I just read a piece by Bambi Francisco (it's here, scroll down in the story) in which she claimed in passing that typing "ugg boots" into the main page at Google returned a subcategory of results - above the regular "pure" search results but distinct from the advertising - called "Product Search." I tried it in quotes - "ugg boots" - and it didn't work. Without quotes - ugg boots - it did. I then tried "digital camera" and there it was again.

What do you know - a new class of results (from Froogle, see my earlier post which missed this) nested rather uncomfortably between the sponsored results at the top and on the right side, and the pure results below. Also interesting was the fact that the results were in the same white background as the regular "pure" search results, though they are distinct in look and feel.

This is a very interesting development with significant implications. Many would claim this is a major step toward Yahoo or MSN-like approaches on the part of Google. Others would argue that Google in fact is simply trying to do its users a favor, inferring that most searchers who type such queries are in fact in shopping mode. But it is a clear departure from the conceit - and I use that term nuetrally - that Google has always maintained, which is that the results offered by their engine are free of human intervention - that they in fact reflect the results of a carefully tended algorithmic secret sauce applied to every site without bias. Clearly, humans have decided to put that category of Product Search on top of the main results. And certainly those results are not subject to the same secret sauce which sifts the rest of the unwashed web. (One can imagine merchants racing to game Froogle, now that Froogle results are showing up - in first position no less - on the firehose of traffic that is the main results page of Google). No matter how you slice it, this marks a departure.

Interesting indeed. I am attempting to find out if this is just a test, a trial balloon of sorts, which Google does do quite often, or if this is a permanent feature. Stay tuned.

Update: Google's David Krane, who runs corp. communications, responded to my query early this morning. He said that the incorporation of Froogle results into the main search page is "a test of sorts, yes...just an opportunity to further integrate Froogle with what folks are looking for on Google. It's been up for a couple of weeks. We'll have more data on the response after the new year."

He offered to have me speak with the product manager in charge of this, and I will be talking to her at some point in the near future, give or take a Christmas holiday. Thanks for the quick reply, David!

HC Intl. Inc., a provider of "business information through trade catalogs and yellow pages directories, search engine services, and television and print periodicals in China" went public Weds., and shares were nearly 10 times oversubscribed. The IPO roared out of the gate, and closed 34% higher than its opening, on a weak market day. IDG Ventures, based here, held nearly 20% of the company pre-IPO.

MediaPost reports that top members of the team behind Sprinks, which Google recently purchased and essentially folded into AdWords (why?see here), have landed at Kanoodle. Kanoodle also got a round of funding, and opened an office in NYC. They have a keyword search product, and will probably add contextual and local advertising products next. It's a sign of the robust market around paid search, and a good thing that companies besides Overture and Google can continue to thrive.

Amazon is not the only one experimenting with making print available online. This is a Big Deal....

If you've ever driven down I-880 coming from points North toward San Jose, noticed the line of homes etched into the Eastern hills above Fremont, and wondered - "Who lives up there?" - the answer is Jakob Nielsen. Nielsen is a reknown user interface expert who's written numerous books and articles on web design, and who was an early Google advisory board member. I drove up to see him yesterday on my way down to the Google holiday party (well, the Google holiday party for members of the press, anyway). Jakob's house affords a sweeping view of the bay area from a more southern perspective, which is a bit disconcerting for someone who is used to seeing the bay from Mt. Tam or UC Berkeley.
Jakob has plenty to say about the state of search and design on the web. When I asked what was next in search, he said "Solving your problem, as opposed to finding you the best site." An interesting insight. He believes search is a critical thinking skill that should be taught in the elementary school system, and I agree. It's amazing how much smarter you can be online if you know a few basic search skills - use of quotes, and/or operators, a few basic syntax elements.
After Jakob I drove across the Valley to Google's new building, this time for a holiday media mixer. Just about everyone who's ever covered Google was there, from Dan Gillmor (SJMerc) to Fred Vogelstein (Fortune) to Kara Swisher (WSJ) to Stephanie Olsen (Cnet). And senior Google folks turned out in force, though some were obviously uncomfortable with being in a room teeming with journalists. I saw about ten folks I have interviewed for the book, it was good to reconnect. All in all a nice affair, and I had some interesting conversations with Craig Silverstein (employee #1), Krishna Bharat (created Google News and is now going to India to help start that project), Shona Brown (new at Google, running biz operations, wrote a good book back in the early bubble that may as well be a blueprint for Google right now), and many others. Larry, Sergey and Eric were there as well, they were predictably mobbed. I said hello and moved on. I hear Cory was in the house, though I didn't see him, but Joi Ito was there, looking younger than when I first met him at Wired in 1993. How the hell did he do that?

While at Wired, we used to joke that we'd know our time was up with Time Magazine did a major feature on us (they did, and it was, sort of - it marked a passing of our hype-driven era). Louis used to say he knew the hippy movement was through when it made the cover of Time.

Well Saddam made sure that Google wouldn't make the cover this week, but the magazine nonetheless managed a pretty comprehensive piece of its own. While much of it is retread, I have to give the author credit - it's one of the first, besides a nice piece in Wired last Feb, which gets to some of the JAM (joints after midnight) issues surrounding search. To wit:

"But for a minute forget about the big numbers, the millions of customers and the billions of dollars. Think about what's at stake culturally and socially in the search wars, and all those zeros start looking pretty paltry by comparison. The Internet is swiftly becoming the primary repository of the bulk of human information. Search is the way we get at that information, and companies like Google wield enormous power. They reflect our common interests and shape how we learn about the world with their rapid-fire search results. This isn't just about dotcom juggernauts duking it out for stock options and bragging rights. Whoever wins the search wars owns the keys to the kingdom of knowledge. That's a big responsibility. Are search engines up to it?

This is one of the bigger questions of my book (not only specific to Google, but to the database of intentions...).

Nonetheless, the media virus that has consumed nearly every outlet w/r/t Google may be played out, now that Time has had its say. Until the IPO, of course....

Alex Salkever of Businessweek gives Google an interesting business model once over, leading with a scenario in 2006 where Google runs just about everything web-related. He introduces the term "Googlization" - which I take means the creeping (his word) dominance of Google over nearly all forms of informational commerce on the web.

Salkever points out that all the new features Google is adding end up stealing traffic and revenue from other companies, starting with the example of Google's new ability to track FedEx packages:

"Perhaps more important, Google is providing this new shipment tracking service even though it doesn't have a partnership with FedEx. Rather, Google engineers have reprogrammed it to query FedEx directly with the information a user enters and provide the hyperlink direct to the customer's information.
No doubt, this is an ingenious way to keep people at Google longer. By extension, the search giant can create more online real estate to sell ads on. But with every new service, Google takes a slice of someone else's pie. Its ability to find pizza places within any given Zip code ultimately eliminates the use of YellowPages. Using it to find word definitions diminishes the business proposition of online dictionaries."

What's interesting here are the assumptions regarding Google's motivations - that the company is entirely motivated by the desire to garner more "real estate" against which it can sell ads. A reasonable assumption for a business magazine to make - that a company is being driven by the motivation to make more money. But that's quite distinct from what folks at Google state they are doing - leveraging searchable data to make a consumer's life easier. "Can't the two co-exist?" I can almost hear them asking. "Doesn't one (profits) follow from the other (helping consumers with a great service)?"

Well, yes, and with AdWords, certainly that was the case. But Salkever further paints an interesting scenario in which Google decides to add a feature where users type in "movies" and their zip code, then get Moviefone.com's results. He concludes:

"Google wouldn't really need to ask MovieFone's permission because the service is publicly available and easy to program into Google in the same manner as the FedEx service.
Yes, such a Google service would probably mean extra traffic for MovieFone.com. And that would be good for MovieFone, right? Perhaps not. The interaction takes from MovieFone at least one prime advertising asset, the usual initial visit to MovieFone's home page, and instead awards that impression to Google. That's because the movie-listing seeker will skip right to the MovieFone hyperlink with their Zip-code-specific information. And as more traffic arrives from Google, that would give it more leverage to ask MovieFone for a referral fee. That's not hard to envision if Google become a public company with a responsibility to, above all, boost profits."

But as seductive as the reasoning is, I don't think Google will sell referrals. Google will think twice, and maybe more than twice, about implementing such a business practice. That would make them a player in the buying and selling of pure intent - as opposed to the paid intent they so assiduously separate from their natural search results. Therein lies a significant conflict. Imagine if they did try to sell those referrals to Moviefone, and Moviefone told them to pound sand. Now what would they do - cancel the service? Keep the service but route the traffic to a different movie service, one that was willing to pay them (or pay them more)? Remember, this is within their pure, untainted-by-commerce natural search results. Maybe MSN would cut a deal like that, but such a scenario feels distinctly "unGoogly" to me. Google's entire reputation is staked on the purity of their search results.

I also don't buy into the idea that Google, or any portal, will become a consumer's sole point of entry and/or query to the web. First of all, there's too much context loss. Consumers like to have context surrounding their interactions with specific services. Second, there's syntax overload. It's amazing how powerful Google and other engines are, but more than 95% of all searches never take advantage of advanced search features. I don't want to have to remember the right syntax to get movies for my zip code, or to type in a particular syntax to get my Fed Ex package. Dedicated sites do this for me in an intuitive way with immediate options in case I get confused. Third, as a competitive differentiator, it's relatively easy to copy - Yahoo, MSN, and anyone else could add these features quickly if they started to gain traction. And lastly, if Google does become a standard referrer, Moviefone and others would simply have to adapt - it's not the first time content sites have depended on portals for referrals, and fought with them about who is delivering the true value to the consumer. It's AOL in 1994, Netscape in 1996, and Yahoo in 1998 all over again. The web will endure.

An interesting interview in Ad Age with Don Logan, Chairman of Time Warner's Media and Communications Group (he runs the Time Inc. and AOL businesses, among others). He's an old Time Inc. guy, and that company has been running out of steam of late, reporting its first ever quarter without profit growth. So he's got two "troubled" units. He speaks about a number of issues, including putting Time Inc. content behind the wall at AOL, launching new magazines, and whether or not AOL should be sold (not if it remains a "sustainable business," hmmm...). (link via Rafat Ali, thanks).

One of the most talked about companies in the paid search analytics business is Go Toast, which makes a business of helping companies optimize and analyze their paid search campaigns. I always expected they would be in line for acquisition, perhaps by a large agency like IPG or Omnicom. Instead, Go Toast has been purchased by Atlas DMT, a unit of aQuantive (AQNT), which is a public roll up of several online agencies and services, including longtime survivor Avenue A.

Foo was fun, and it also showed that optimism was back in the geek inner circle.

THE MESSAGE
The Geeks Are All Right
What happens when 200 hackers and visionaries camp out in the hills of Northern California? If you have a stake in the future of business, you'll want to find out.

By John Battelle, December 2003 Issue

Stashed away in the rolling hills north of San Francisco, the town of Sebastopol, Calif., used to be remarkable for two things: Gravenstein apples (it was once the world's largest producer) and the Russian River appellation (excellent zinfandels). You can now add a third important growth industry whose roots are there: Foo Camp, a new breed of geek gathering organized (somewhat) by O'Reilly & Associates, a thriving technology publishing business.

This year's Foo Camp, held in early October, was extraordinary for many reasons, but perhaps mostly for its structure -- or lack thereof. Tim O'Reilly, Foo's founder, made sure that basics like food, showers, and meeting space were available, but then quickly turned over the weekend's agenda to the geeks (literally -- there was no agenda until Friday night, when the attendees made one up on the fly). The idea: Get 200 or so smart folks with a lot in common together in one place at one time, let them pitch tents, toss in a Wi-Fi network, and see what happens. Turns out, quite a lot. (more via link below)

Continue reading "#9: Springtime in Geekland" »

I do believe there is a huge market to be made in adapting video to weblike channels...

THE MESSAGE
Gone in 30 Seconds
The classic TV spot can't dominate advertising much longer. Microsoft's new MSN Video is bringing Web-based commercials to a computer monitor near you.

By John Battelle, November 2003 Issue

NOTE: This has been updated from the version published in the November 2003 issue of Business 2.0

Let's consider a few fun stats. First, broadband has reached nearly 39 percent of all Internet-connected households and is expected to be in 79 percent in five years. Next, research firm In-Stat/MDR predicts that corporate spending on broadband content-delivery services, essential to high-quality video streaming on the Web, will almost triple by 2007. Third, the vast majority (72 percent) of work connections are high-speed. Throw in the forecast that TiVo-like devices will be in 20 percent of American homes (and certainly a higher percentage of wealthy ones) by 2007. And finally, 99 percent of homes with incomes over $100,000 have Net access, and it's a safe bet that most of those have broadband connections.

Add it all together and here's what it means: A massive broadband marketing opportunity is upon us -- and, as a recent Yankee Group report concludes, the 30-second TV ad will soon lose top billing as our most valuable marketing vehicle. What will replace it? Web-based video advertising. (more via link below)

Continue reading "#8: Intent over Content, Fish with Feet" »

A longer piece on MSFT which came out of the news that Yahoo bought Overture.

THE MESSAGE
Mr. Gates and the Hunt for Search
Yahoo's grab for Overture seemed to outflank Microsoft. But wait: Bill's search-engine strategy is not what everyone thinks.

By John Battelle, September 2003 Issue

Like TV broadcasting and the automobile business before it, the Internet media industry has now resolved to the Rule of Three. Of the scores of companies that battled for dominance in the 1990s, only Yahoo (YHOO), Microsoft (MSFT), and Google remain serious contenders. They, along with AOL (AOL), own the lion's share of Internet advertising and worldwide English-language traffic. Yahoo's recent acquisition of Overture (OVER) reaffirms, if the idea needed reaffirming, that Internet media obeys the same urge to consolidate as every other industry.

It proves something else, too, about this phase of the Internet's evolution. The key driver is no longer content, but intent. The business is no longer about selling advertisers the eyeballs you've caught with news, images, games, and the like. It's about selling users at the moment they make their online desires known through their search queries. In plain terms, the engine of Internet media is once again search. (That, by the way, is why I'm leaving AOL out of this discussion. Preoccupied with many other problems, the dial-up giant -- a corporate sibling of this magazine -- is leaving its search functions to Google.)

Search will account for more than $2 billion in advertising sales this year. It's predicted to grow at 35 percent annually, to nearly $7 billion by 2007, according to U.S. Bancorp Piper Jaffray. Beyond the numbers, search has become the most important commercial application on the Web. Not only is it the defining task of any portal, but it's also the preferred doorway into e-tailers like Amazon (AMZN), as well as auto, home, and dating sites. An online consumer business can no longer afford to have poor search capabilities.

By the middle of last year, both MSN and Yahoo had realized that they needed to rethink their search strategies. To profit from search, a company needs three elements, all of which Google already had. First, you must have high-quality "algorithmic" search, which attempts to match users perfectly with what they're seeking. For years MSN and Yahoo have outsourced algorithmic search to companies such as Inktomi and Google. Second, you need a paid search network, which allows you to display links to paying advertisers alongside your editorial results. Both MSN and Yahoo had outsourced this to Overture. And third, you need your own distribution. In other words, you must own the site where the consumer makes his or her query and the results are displayed. Until recently this was the only element that either Yahoo or MSN truly owned.

By buying Inktomi last December and then Overture in July, Yahoo has taken control of the two missing elements, which arguably leaves MSN in the worst position of the Big Three. Besides having the only site of the three that is consistently unprofitable, it is outsourcing both its algorithmic and its paid search technology to a major competitor. Not exactly an ideal situation.

But aside from Netscape's early investors, not many people have ever gotten rich by underestimating Microsoft. (more via link below)

Continue reading "#7: MSFT and Search" »

Think you control your own entertainment system? Think again.

THE MESSAGE
Don't Stifle Progress
A number of states are trying to do what the federal government couldn't: Kill innovation on the broadband Internet.

By John Battelle, August 2003 Issue

What if it were illegal to connect your computer to the Internet without asking permission? It seems unthinkable, but an ongoing legislative imbroglio has brought that very question to the fore.

The fight pits the cable industry and (surprise!) the Motion Picture Association of America against a battery of consumer and civil liberties groups. It revolves around a set of bills that opponents have labeled "Super DMCAs" -- state-level legislation modeled on the much-reviled federal Digital Millennium Copyright Act of 1998.

During the past 36 months, these bills have been passed in the legislatures of seven states, including Michigan and Pennsylvania, with nine more waiting in the wings. In each case, the legislation is cloaked in the seemingly benign language of preventing cable signal theft and has been quietly fed to a somnambulating legislative body. But all of these bills also contain extraordinary language that gives the access industry (cable, satellite, and phone) complete control over what devices you can connect to their broadband networks. They also make many things we take for granted -- such as anonymity, firewalls, even instant messaging -- potentially illegal.

In other words, these bills could create "early lockdown" of the broadband Internet, meaning that every aspect of how the network is used could be dictated by the access provider. And when the rules governing a robust system prevent innovation and favor one set of players above all others, the system withers. Even the Internet.
(more via link below)

Continue reading "#6: A Growing Early Lockdown Threat: Super DMCAs" »

Open source search - in an age where innovation is increasingly silo'd in large companies, this is a good idea whose time has come. I didn't like the "watch out" angle, but...it gets a reader's interest.

THE MESSAGE
Watch Out, Google
Nutch could rewrite the rules of search development -- especially with an impressive roster of Internet luminaries now lining up behind it.

By John Battelle, August 08, 2003 (Web Special)

Ask anyone in Silicon Valley what the hottest application on the Internet is today and you can bet their answer will be search. The dealmaking has been nothing short of torrid. Only a year ago there were at least half a dozen major players. Now there are just three: Yahoo (YHOO), which last month bought search giant Overture (OVER) in a $1.6 billion deal; Google, the undisputed king of search; and Microsoft (MSFT), which is busy building a search platform of its own. They're all fighting to dominate the huge and ballooning market, already worth $2 billion and expected to generate between $6 billion and $8 billion in revenues by 2007.

Search is a game of intellectual property, innovation, and market position. The three combatants all keep jealous watch over their patents (Yahoo, for one, has more than 60), engineering talent (hundreds of Ph.D. holders work at Google), and market advantages (Microsoft -- need we say more?). Indeed, search is such a complicated and expensive undertaking that analysts have pegged the cost of market entry at well over $100 million.

All that could change this fall, when a new player strides onto the field.
(more via link below)

Continue reading "#5: Nutch Presages a New Kind of Search Engine" »

When I saw this ad, I knew I was watching the future of online video advertising unfold.

THE MESSAGE
Downloading the Future of TV Advertising
With a plink and a plunk and 86 moving parts, Honda reminds the ad world of the value of great content -- and teaches it something about the power of interactivity.

By John Battelle, July 2003 Issue

In April 2003, Honda U.K. debuted an extraordinary two-minute television advertisement called "Cog."Through the simple act of releasing a remarkable television commercial onto the Web, the U.K. wing of automobile giant Honda (HMC) has unleashed something of a typhoon in the advertising business. Though it has yet to fully play out, Honda's ad proves the value of content and could stand as a turning point in the history of the television spot -- proof that interactivity won't kill television advertising, as many are now predicting, but may instead be instrumental in saving it.

Back in April, Honda U.K. debuted an extraordinary two-minute television advertisement called "Cog." Aired only in the United Kingdom, the film -- and that really is the best term for it -- is a Rube Goldbergian ballet, a synchronized dance of 86 distinct parts from a Honda Accord that roll, pirouette, and fly along the floor in a mesmerizing production of meticulously intended consequence. The spot begins with a sequence of three cogs rolling along a plank; one falls to the floor, and a cam shaft rolls, setting an exhaust tube slowly spinning, which knocks three precisely placed grommets down the slope of a hood, and so on. (Download the ad for yourself at Honda UK.)
(more via link below)

Continue reading "#4: That Wonderful Honda Ad" »

Oh, if only we had this three years ago, at thestandard.com...

THE MESSAGE
Putting Online Ads in Context
Overture and Google have figured out how to sell the Web. Paid search has already saved Yahoo -- and your business might be next.

By John Battelle, June 2003 Issue

The long-awaited "30-second spot for the Web" -- a way for ads to finally work online – may well be at hand. Overture (OVER), the company some say saved Yahoo's (YHOO) bacon, will shortly roll out a service that opens up the entire Web to a new form of advertising. "It's potentially revolutionary," says Scott Moore, who oversees Slate and MSNBC.com for Microsoft (MSFT). How revolutionary? Moore says using Overture's new service, or one like it, could well push his sites to sustained profitability.

The breakthrough, which I'll call "contextual advertising," involves commercial links that appear adjacent to relevant content on websites. Say you're at Caranddriver.com, reading a review of the Acura MDX. In place of banners for everything from cell phones to cars you don't care about, you would see paid text links advertising the Acura website, the Edmunds auto comparison site, and leasing companies vying for your business. These are the same links you'd see if you typed "Acura MDX" into Overture's client portals, like MSN or Yahoo.

Google also plays in this new market with an offering called "content-targeted advertising." The beauty of both is their ease of use for publishers: Overture and Google automatically analyze the publishers' pages and insert relevant links on the fly. All the publishers have to do is collect a check. It's close to manna from heaven.
(more by clicking on link below)

Continue reading "#3: Why Contextual Paid Ads Matter" »

For this piece, I daydreamed about how TiVo might be saved by Steve Jobs. I wish this were the case. With cable coming on strong, I am not feeling so cheery about TiVo's chances.

THE MESSAGE
Is TiVo NeXT?
The beleaguered personal video recorder company is ripe for an Apple takeover.

By John Battelle, May 2003 Issue

Everyone who has TiVo (TIVO) loves TiVo; it is to television what Macintosh was to computing -- a revelation. Which is exactly why Apple (AAPL) should buy TiVo and once again redefine the intersection of culture and technology.

Folks love TiVo for the same reason they loved the Mac in 1984 and the iPod in 2001: It gives control back to the end user. TiVo viewers call the shots regarding when, how, and -- soon -- even where they watch. Once content or access is purchased, the end user is in charge, just like with the iPod.

But unlike the iPod, TiVo and systems like it are in serious trouble. The culprit is the entertainment industry. TiVo has an abeyant Napster-like quality -- and the content business is scared silly that it will not only destroy advertising revenues but become the platform for video swapping on the Internet. Case in point: A coalition of entertainment companies recently sued TiVo competitor Sonicblue into bankruptcy.
(more by clicking on link below)

Continue reading "Piece #2: Please, Steve, Buy TiVo" »

I wrote this a year ago (there's a three-month lag from writing to pub date with monthly magazines). I was sick of the press hammering the internet, with the presumption that everything we did over the past four years was a waste. Turns out, it wasn't.

THE MESSAGE
Alive and Well
From content providers to dog-food retailers, Internet business has moved to a new stage of stable growth. That's the real story -- so why aren't we hearing it?

By John Battelle, March 2003 Issue

Any avid reader of the business press has seen endless variations on this tired theme: Internet business is dead. It was all a dot-con, and it dot-bombed.
Eager for an easy target and brimming with schadenfreude, many business reporters (and a few opportunistic book authors) continue to tear down the Net with nearly the same enthusiasm they displayed while building it up. But the facts tell a different story.
Let's start with the flashing VCR clock of all dot-bomb maxims, the Internet pet-food industry. There's no better proof of dotcom stupidity than the fact that venture capitalists funded not one but at least four pet-food websites at the same time. Thank God for us all, they are dead.
Except ... they're not. In fact, type "buy pet food" into Google and you'll get scores of active merchants selling pet food online. I put in a call to one of them, Geoffrey Walker, CEO of PetFoodDirect.com. Surprise: His business grew 22 percent last year, and he expects similar growth this year. In fact, he and his three or four biggest competitors -- yup, there are still that many players in this category -- are all doing well. As for Pets.com, the now-defunct Sock Puppet company, Walker is thrilled about all that exposure, which let consumers know that they no longer had to lug around 40-pound bags of kibble. Pets.com now redirects to Petsmart (PETM), whose stock price has rung up a 44 percent increase since a year ago. Woof! (for more click link below)

Continue reading "My First Piece in 2.0" »

Yeah, I've been threatening to post the past year's Business 2.0 columns, and I'm about to start. I've decided to post then in sequence as full text posts to the home page, then classify them as "Columns" using Moveable Type's "Categories" tools. That way, once they've moved off the home page, they will live forever under a "Columns" link on the left. This way, future columns can be posted and added to the list automatically.

(As long as I'm posting site notes, if you can't see a picture in the upper left hand corner, make your browser wider by pulling the lower right hand corner to the right till the picture shows up....ahh, that's better).

So take note, a blizzard of posts (well, about a dozen or so) will follow, and you may want to simply ignore them for the next little while.

SearchDay summarizes Google's recent update to Froogle. The update is not a total redesign, nor a clear statement by Google that it wants to jump in full throttle, like Yahoo has (Google has yet to take off the "beta" moniker). Shopping engines in the main present something of a challenge for Google - by their nature they invite commercially-driven presentation of results, and Google is best known for its "pure" or "natural" results. As Ninj pointed out to me last week, at Yahoo Shopping they believe that shoppers *expect* to be marketed to while in shopping mode - the metaphor she used was Yellow Pages. Those who paid more, have more credibility and larger ads, and that means something to shoppers. Such an approach is not exactly dead center to Google's overall brand values, but neither does it directly contradict them. It is worth noting that the company updated Froogle in time for the holiday season and has been promoting it for several days in a row. To wit: the phrase "Shopping? Try Froogle - Google's product search service" appears on the main page. Google product managers tell me it's extremely difficult to get a promo for anything onto that home page, as they count every bit on that page, each additional bit slows down performance. Clearly, a decision has been made that this is important to the company, and by extension, to the company's users.

That's what John Chambers of Cisco dubbed our nation's youth back in the bubble era. Well, now that the internet is cool again, USA Today has come up with another moniker: The Google Generation. The author of this article, a child psychologist, makes a good point: "As members of the Google generation, today's children have facts at their fingertips. They don't need information fed through toys. They need to play and to become creative problem solvers." I certainly agree with this. A quote used by the director of my kids' school (originally from Socrates, I think) goes something like: "A child is a flame to be ignited, not a vessel to be filled." She suggests holiday toys that do not entertain, but rather that provoke creative play. Her top ten: Play-Doh, building blocks, costume drawers, puppets, red rubber balls, books, crayons, paints, rhythm instruments and dolls.

I spent most of yesterday working on the book, talking to folks at Yahoo, Google, and with Paul Saffo. I met with Terry Winnograd, who was Larry Page's professor at Stanford, worked with Larry and Sergey early in the project, and is still on Google's technical advisory board (scroll down). While I was there I ran into Yossi Vardi, who is always a joy to see. The man is always beaming, and this was no exception, he was touring Google's new building with Sergey and telling enthusiastic stories about Sergey's recent trip to Israel (Yossi of course played host). In any case, internationalization is clearly a theme at Google. The company recently announced a new R&D center in India. I asked David Krane (Director of CC) why, and he said he company can't expect everyone to come here, and there's a lot of talent in Bangalore. Makes sense.

Terry is an energetic man, he was a founding member of CPSR and has done a lot of work on natural language and HCI (human computer interface). He recently took a sebbatical from Stanford to work at Google full time, and is now moving back into the academic life. I won't go into all we discussed (gotta save something for the book!) but it was a good meeting, and I heard some funny tales of the early days, among other things.

At Yahoo I met with Srinija Srinivasan, Yahoo's Editor in Chief. She's been there almost since the beginning, and we had a robust and lengthy conversation about the role of editorial in search, the role of the directory at Yahoo (very interesting) and challenges ahead for the industry and the company. I really enjoyed Ninj, as she is known. She's only 32, but has spent nearly 9 years thinking about this stuff as EIC at Yahoo. Prior to that, she worked on Doug Lenat's CYC project, which for those of you unfamiliar with it was a very brave, arguably foolish, and ultimately unsuccessful attempt to solve AI's "brittleness" problem. CYCorp still exists, applying some of its early work to corporate data issues.

Lastly, I spent the early evening with Paul Saffo, who is a walking idea machine. Paul is something of a metaphor graffiti artist - he could earn a living tagging metaphors across anything that catches his fancy - wait a minute, that *is* how he makes a living. He came up with a few really good ones as it relates to the Database of Intentions and various other Big Ideas in my book. Hint: Think Middle Ages.

Marchex, a Seattle-based SEO/Pay per click firm headed by folks from Go2Net, has filed to go public. Here's a story on it. This is interesting in many ways - the company is not profitable (though it is cash flow positive), and it's not well known, but it's in the search/advertising space, which is feeling overheated. Worth watching.

One of my more inchoate but deeply felt rants has to do with the role of video in our culture. I'm convinced, for reasons I can't properly articulate, that as a culture we've hobbled ourselves by refusing to make video - in particular the incessant stream of television so omnipresent in our lives, a more citation-friendly, searchable, and conversational medium. What I've always wanted was the ability to approach video much as we now approach text - it can be searched, annotated, cut and pasted, linked to, etc. I want to be able to say "Hey, remember that great rant by Jon Stewart on Halliburton?" and then link to it or email it to a friend. I hint at some of this in various 2.0 columns. Of course there are many technical and legal issues with the implementation of such a dream. Fist, bandwidth is still too expensive for most mere mortals to be hosting massive libraries of video. Second, the numbnuts at the MPAA. And third, video must be logged and tagged to be searched - it's not a self-tagging medium like text. But there is hope. For issue one, there's the optimism (if not the politics) of folks like Gilder. For issue two, there's folks like Larry Lessig. And for three, there's closed captioning (it's a start!), and the work of lesser known but really exciting companies like ShadowTV (thanks for the link, Gary!).Worth grokking, and good to know smart folks are on the case here.

So the debate on re-naming, or re-spinning, or re-thinking what RSS might mean to a broader world is playing out, at Contentious and on Dave Winer's blog as well as at Salon and here. But as I've been thinking about it, I'm increasingly convinced that the phrase we're looking for we already have - The Web. That word can shapeshift enough to incorporate the changes inherent to a pubsub world, as Dave puts it. Maybe what we're really looking for is a better verb. Let's kill surf, as soon as possible. And come up with something better.

* Brainboost, an Ask Jeeves like Q&A engine (employs "cutting edge AI!")
* Think Tank 23 (finds "ideas")
* Nervana. " You define the context of what you need, and Nervana delivers semantically relevant results from a multitude of sources (such as documents, emails, knowledge communities, intranets, and the Internet). " (via Gary Price's Resourceshelf)

There must be hundreds of these small search-related companies already, with scores more on the brink of launch.

And the Top Ten are (with last year's ranking in parens):

1. KaZaA (2)
2. Britney Spears (4)
3. Dragonball (1)
4. Paris Hilton (-)
5. IRS (7)
6. Kobe Bryant (-)
7. Christmas (9)
8. NFL (6)
9. Pamela Anderson (10)
10. Brooke Burke (34)

Check out the various slices and dices in Lycos' version of the Database of Intentions...

I've not fully grokked it yet, but here's a new concept in search: Eurekster. From the site: "Eurekster uses the six-degrees of separation concept to learn from your extended network of contacts and deliver you prioritised results based on the success and proximity of the searches they have done." The site is in beta and you need a password to get in, a friend who will remain anonymous gave me his. I signed up to be official, we'll see if I get the nod.

Meanwhile, I see the folks behind this are RealContacts and SLI Systems. SLI is the tech that was behind Snap!, and survived it (and now powers some portions of the NBC webspace).

I can't tell is this is simply silly - take the hottest stuff on the web right now and combine it! - or for real. More when I get smarter.

So last night was the launch soiree for version 2.0 of Grokker, a search interface tool built right in my own backyard of Marin. It's backed by Paul Hawken, and sports some impressive advisors, investors, and board members, including Paul Saffo and John Seely Brown. Grokker "gives the big picture" on large collections of data - it's a visualization tool, and from what I saw last night, the new version is quite elegant. I saw a demo of Version 1.0 about a year ago and they've really made a lot of progress. I am a huge fan of the word "grok" - from its original Heinlein origins through Wolfe and Wired (where I helped the author of Wired Style write the definition) and the Standard (where we launched a series of newsletters and a magazine called "Grok"). It's neat to see it making another round through the vernacular.

In any case the party was quite subdued by usual product hype standards, with an odd assortment of well-to-do investor types wandering shoeless through an extraordinary Mill Valley home, and Bonnie Raitt also in attendance (Bonnie Raitt?!), sporting a very down-to-earth and pleased-to-be-here manner. Groxis CEO RJ Pittman gave me a demo. Grokker takes datasets created by a keyword or phrase queries on sites like Amazon or "the Web" (it hits six search engines) and runs them through a second filter which displays results as clusters of nested orbs, each with tags derived on the fly from the data. It's quite a seductive interface, and I can see it working for any number of search needs. And as a trend, I applaud this kind of development - building new applications based on search as a platform, rather like an OS. (Tim Bray among others have noted in the past how search today seems stuck at the level of the DOS C: prompt. Groxis might be seen as an attempt to go GUI.)

But the most interesting thing about Groxis is how it is approaching versioning - this release comes after more than a year out in the open, soliciting feedback (some of it quite negative) from the search community. With Version 2.0, Groxis plans to again listen and learn from how the product is used, rather than try to force it into a particular bucket of revenue (though they do have enterprise and licensing deals). This is due to the angel investors behind the company, Pittman said, who are not demanding a rigid, pre-determined approach to how the product will make money. That was exactly how Google did it in the beginning (and, it seems, how Friendster is doing it now). The Mac version will be ready to beta in a few weeks. When I get it, I'll post more.

Last night at the Webby Business Awards dinner (caveat, I was a judge), while having a lively conversation about the future of technology and business online, it struck me that it might be interesting to ask the folks at the table not what they thought was coming, but rather where they were three years ago. Because, as it happens, three years ago this month was pretty much when the worm turned in the internet industry. Late 2000, the money ran out, the businesses started to fall apart, the media coverage got bitter and angry, and a lot of people started losing their jobs. Not surprisingly, most of the folks at my table had entirely different jobs three years ago, as did I (there were folks from Google and Salesforce present, and as you might expect, they were in fine fettle).
Three years ago I was running the Standard, facing the first layoffs in the company's history, and dealing with a dramatic reversal in our advertising forecasts: a drop of about 75%. So where were you, and what were you doing? And...more importantly, how are you doing now? Are you happier? Interestingly, every single person I asked that question last night said they were. Worms keep turning, and eventually the soil again bears fruit.

I'm scanning my media news this morning and I see an item titled "Cablevision to Offer Tivo-Like Service." Sigh. It's so sad to watch a great idea, and a pretty good execution, fade right in front of your eyes, as pale imitators create "Tivo-like" services.

The problem with this, as I've ranted before (in a previous 2.0 column that I swear I am about two days from posting here), is that while TiVo puts its intelligence at the edge of the cable network, ostensibly under your control, the cable company boxes have opaque, client-server architectures, owned and controlled by the cable system. In other words, they decide what you can fast forward, record, cut and paste (well, you can't do that anyway), etc. It's a terrible, closed system, and because cable companies are monopolies, for the most part, there's not much to be done about it. It's Larry Lessig's worst nightmare (and mine too).

And how might the cable companies handle their newly found control over your PVR? Well, here's a hint from Tom Rutledge, president of cable operations for Cablevision: "We didn't think it was a good idea (to add PVRs to the system) and didn't want to do it," Rutledge said. In other words, we are with the MPAA and the content industry on this one, and are only doing this because the market is asking for it. Endgame: If you get a cable PVR, expect the keys to be in someone else's hands. My suggestion: go satellite and buy a TiVo. This year might be your last chance.

Clearly Searchblog's most popular post of all time (OK, so we're only a month old) is this one, in which we attempted to take Scott Rosenberg up on his challenge for a better name for RSS. There is even a contest happening over at another site, and I am sure this meme will evolve for some time to come. So far the conversation has included names as far fetched as "chunkyweb" (the web was plain, RSS makes it chunky), Feedcast (which I rather like), Flow (which prompted some interesting comments about gushing loins), Readster (more of a product name), and a thought that perhaps what we are really looking for is a better name for the Semantic Web. I say, keep on talking. It's fun to whiteboard ideas this way.

An interesting coup for Cnet: MediaPost reports it topped all other search engines, by a rather wide margin, in a recent WebSideStory study which compared vertical shopping engines on a key metric: whether visitors from the engine who clicked through to a commerce site actually convert into buyers. Cnet's conversion rate of 1.64% was better than AOL, the #2 engine, by 28%.
I'd like to know why this is. I've sent a note to folks at Cnet, and I congratulate them - it's been a rough few years for them as they weathered the tech nuclear winter, but I've always had a soft spot for the company - it launched roughly at the same time as Wired, and has really seen it all.

Order Conversion Rates: Searches To Gift/Electronics Sites

Conversion Ratio
CNET 1.64%
AOL 1.28%
Overture 1.25%
iWon 1.15%
LookSmart 1.14%
MSN 1.04%
Yahoo 1.01%
Netscape 0.93%
Lycos 0.92%
Google 0.85%

Update: I must've been rushed and didn't think this through, as I was driving to a Webby business awards tonight I realized of course Cnet has better conversion because it's a vertical portal that aggregates folks with intent to buy particular technology items. Matt points this out in the comments section as well.

Google has landed a deal with the CIA for its enterprise search solution, Government Computer News reports via IESDB. I hope this doesn't meant the CIA will next set its sights on Google's version of the Database of Intentions....

.
EWeek reports that a new software application is now available that blocks paid search ads on most popular search engines. I wonder if this will take off, or is ephemeral? It depends on how consumers react to the next wave of search engines - how Yahoo and MSN do their next implementations, for example. I don't think anyone really objects to how Google does ads right now, in fact, I think most feel the ads are even useful. I'd be curious to know if any readers are using this product and if so, if it in fact is useful. For more on Intermute, the company making this software, here's their site.

Tim Bray has a series called On Search over at his Ongoing blog, and I find it worthy of a read'n'muse. He starts with this backgrounder on himself and search issues as he sees them, and has a ton of entries on any number of subjects, too numerous to go into here. Highlights: he writes on interface issues (warning, not for the faint of geek), how best to search XML (answer: we don't know yet, recall he was a co-author of same), and on result rankings, with a quick refresher on why PageRank works, and good advice on paying attention to your own logs. Also worthy: his primer on how search works, and his discussion of the technical search terms precision and recall (with an interesting note on the absence of top companies in the research community - see my post on this here), and lastly (whew), his mini-rant on intelligent search, and why it's a long way off. An excerpt:
"If we want better search (and we do), we’d better not count on AI voodoo or linguistic juju or semantic mojo. We need to work with good sound statistical techniques, and be clever about generating and using metadata, and we need to get our APIs right. All of these things are hard, and there is good work being done in all of them."

...and with his conclusions, I must say I agree....

This paper "presents a high-level discussion of some problems in information retrieval that are unique to web search engines," according to its abstract in the ACM library. (A reminder as to what this whole "Search Papers" thing is about: read this.) "The goal is to raise awareness and stimulate research in these areas," it continues. How might such a lofty incitement be backed up? Well, it's written by two senior employees of Google, Monika R. Henzinger and Craig Silverstein (I've met with Craig, he was employee #1 after Larry and Sergey, and a nice guy to boot), as well as Rajeev Motwani, a professor at Stanford (Craig was his graduate student).

The paper is dated September, 2002, so it does not rank as a missive from the early, more geeky phase of Google's life, but rather a more corporate product - the two Google authors knew they bore the weight of "being Google" when they wrote this paper, and it's worth keeping that in mind when reading through it.

This is particularly clear in the paper's scope and focus. It lays out six challenges for search engines - and they read like a laundry list of Google's headaches. The paper then goes on to offer suggested paths for more research on the topics, which I could imagine might read either as genuine or a tiny bit patronizing, depending on who you are. (The paper does not tackle a range of other issues it says are already the subject of abundant research - natural language queries, image/audio search, improving text-based retrieval, language issues, or interface/clustering, for example.)
(more in the extended entry, click link below)

Continue reading "The Search Papers: Challenges in Web Search Engines (A Google Paper, 2002)" »

A recent Blog Search Engine survey written up in Marketing Wonk shows that 13% of bloggers run ads, 9% have been "approached by companies to blog about their products" and 7% blog for money. While Marketing Wonk spun these as low numbers, I disagree, I think they are quite high, given the early nature of the form. In particular, the 13% who run ads sounds way too high - I doubt 13% of websites ran ads in 1995, for example. A quick review of the methodology, such that it is, shows that it's a survey of 610 bloggers who have submitted their sites to the Blog Search Engine and "other blog owners contacted through different channels." Come on, we can do better that this! I think these kind of stats are fascinating, and would even be useful if they were in any way defensible as statistically significant.

I spent much of yesterday morning talking with Brewster Kahle, of WAIS, Alexa, and Internet Archive fame. Brewster is a very fun mind, and he's working on about ten Really Interesting Things at once. First, it's easy to forget how important the Internet Archive's work truly is. The public sphere is diminishing as more and more data (in particular log data) becomes owned by corporations, and the archive is one of the few institutions, outside of our often scleortic library system, dedicated to preserving our digital record on a massive scale. (Good quote: "The original purpose of libraries was preservation and access" to society's information, but they've somehow become about "selection and categorization.") He's archiving 20 global television channels as well (see the Television Archive for more.) He's setting up a broadband distributed wireless LAN across San Francisco, and is still cranking out books via the Internet Bookmobile. Brewster showed me the Archive's new "recall" search features, which have been worked up by Anna Patterson of Stanford. Now this is some cool stuff. It searches over 11 billion documents - nearly 4 times that of Google (and they've indexed about 1/3 of what they have). Check it out, and play with the various knobs and graphs. It points to some interesting new concepts in search.

Funny how am idea gathers momentum. As I was penning my Implications of RSS For Business column for 2.0 (awaiting publication in dead tree form in three weeks), Scott Rosenberg was writing a pean as well, published in Salon this morning. He suggests we need a name for what RSS represents, just as the Web became the mainstream's understanding of HTML, we need a name for RSS. He reminds us we've been here before (remember Push? I was a reluctant contributor to this 1996 article, which began as an email thread in the Wired offices...)

In any case, I agree with Scott, we need a name. All the businesses in this space are still in the pre-market phase. RSS allows us to connect more efficiently, to grok information as we like it, when we like it - but what do we call it? I like to say my reader and blogs/news sources is my personal ecology - is there an idea in there somewhere? In any case, it's exciting to see the idea start to take popular flight. Watch for the NYT treatment soon.

Type "miserable failure" into Google. Out pops Google's top pick: George W. Bush. This comes to me via Farber's IP list, but many others have crossed my desk over the past year, including one on "weapons of mass destruction" which - if you hit "I'm Feeling Lucky" - still gaves a 404: Not Found-like error thanks to the vagaries of web humor, linking, and PageRank.

UPDATE: This was a case of Googlebombing, see here....

Today I bounced around a bit, from a very stimulating two hours with Louis Monier, founder of Alta Vista and current head of R&D/Search at eBay, to attending a CMO roundtable discussion at Yahoo where co-founder David Filo spoke, to a Churchill Club dinner in SF where Walt, Kara, Greg Harper and Larry Page talked about their favorite gadgets.
I asked Louis to react to the ideas in my monoculture post - that the best minds are now silo'd in private corporations. He had a great response: "Airlines are not built by the academy." In other words, very complex and expensive stuff by definition is done by the private sector. Monier was, in fact, a veritable quote machine, and were it not so late, and were I not so tired, I'd create a whole post on the subject. Perhaps tomorrow. One more: "Google is, I think, the intellectual descendant of Alta Vista....I always said, with search you need a sharp pencil, that's all." In other words, don't give me more than I need, just focus just on solving the search problem. It is true, Alta Vista was briefly THE search engine that mattered - about 1996-97, before it was ruined by Compaq and later CMGI.
The CMO discussion focused on Yahoo's brand. It featured a panel of senior Yahoo execs talking about what makes the brand special, with an emphasis on how they live the brand from the inside out - how the company and its employees act in a way consistent with what the brand means to the outside world. A bit flat, but there was some neat stuff - I always like to hear Libby Sartain speak (she's the head of HR there) - she's infectious and quite inspiring. David told the early founding story, which was good to hear from the founder's perspective.
The Churchill Club was livelier. There's nothing like geeks with toys to get a crowd going, and Larry for one outdid himself. He really does love this toys, and seemed quite in his element - relieved, I imagine, to for once be in a public speaking role where he does not have to directly bear the weight of being a founder of Google. Cool stuff included a blue tooth ear plug for cell phones, LED flashlights (no bulbs, quite bright, long battery life), a 4 million pixel webcam that is its own POP server and looks rather like HAL from 2001, and touchless, digital toilets from Japan that require no toilet paper (they wash and dry you at the touch of a button. Really.)

So I am offline today (down in the valley meeting Louis Monier and various folks at Yahoo) and I checked into a local Starbucks, and hey, I'm the feed of the day on Feedster! Welcome all of you who might stop by thanks to the pointer. For a tour of what the site's all about, check out this post on the book and for more on me, see this as well.

Thanks Scott!

I've been thinking lately about the role of blogs and RSS in search, and that, of course, has led me to both the Semantic Web and to Technorati, Feedster, and many others. Along those lines, I recently finished a column for 2.0 on blogs and business information. I can't reveal my conclusions yet (my Editor'd kill me) but suffice to say, I find the intersection of blogging, search, and the business information market to be pretty darn interesting.
I'm certainly not alone. Moreover has created "Enterprise-Grade Weblog Search" - essentially, a zietgiest mining tool for corporations. One can imagine similar products from any of the RSS search engines, or even from the major marketing agencies of the world. On the other end of the spectrum (making blogs easier to read for consumers, as opposed to easier to mine for product marketers), Meg Hourihan is fast at work on Kinja, which is going to be a blog of blogs that will make finding and following blogs easier. Bloglines is doing similar work.
What makes this interesting from the perspective of search is the structured nature of what is being searched - blog postings and news articles, for the most part (or maybe I should say the data is "vaguely structured" - a term in a paper I just read - more on that later). Anyway, I sense a pretty potent market shaping up. Anyone know of other folks, companies, or news I should know about?

For those of you who don't follow the vagaries of search engine index updates, the past few weeks may have been pretty uneventful. But for the businesses and marketers who make their living by ranking well in Google's listings, it's been a pretty tumultuous month. That's because once again the Google Dance has swept through the search engine markets, and this last one was unique.

"Google Dance" refers to the process by which Google updates its index - the master code that determines which listings you see as a result of your queries. Because it is a massive index, and because Google often wants or needs to incorporate various tweaks and refinements to its site-ranking secret sauce (in particular to fight spam), it can take days or even weeks for a new index to settle across the web. The folks over at WebmasterWorld track this stuff quite closely, and have taken to naming each update alphabetically, following the nomenclature usually reserved for hurricanes. This past one happens to be F, and has been dubbed (dub-ya'd?) Florida.

When the Google index dances, many sites which once enjoyed top listings can fall off the first page of results, for seemingly arbitrary reasons. Danny Sullivan has written extensively about this, (including today's newsletter) as have many others. What makes this most recent update fascinating is that for the first time, Google watchers have figured out a somewhat consistent way to track what's changed since the last update, and many of them are crying foul. They're claiming that Google has tweaked its algorithms to favor terms which have commercial value, and to prove it, they've hacked up "Scroogle," a tool that shows the before and after results for any given query. Danny's given an interesting example in his article of what happens to the query "laptop rentals" in a before and after test.

While a case might be made, I can't imagine Google is favoring its own commercial interests over its stated philosophy of focus on search, the profits will follow. It'd be suicide for a company that's made its name on "not being evil." The uproar over the Florida update, however, points to the increasing responsibility the company bears as a holder of public trust. Updates like Florida have major implications for an entire economy of Google-dependent merchants, and the two parties (Google and the merchants who depend on its listings) are only growing more interdependent. A dance, indeed.

All the braying about Florida aside, it's interesting to note what you get when you search for "laptop rentals" on MSN. At least MSN is consistent - whoever pays MSN the most, you see first!