John Battelle's Searchblog

I've just finished reading A Taxonomy of Web Search by Andrei Broder, written largely while the author was CTO of Alta Vista (and using AV query data), and published after he moved to IBM Research in 2001.

The paper has a trove of references to other papers, which is good for my work, and it has a singular thesis: that all web searches are not equal. Broder sets out to dispel the notion that all searches are "informational" in nature. He instead maintains that many are "transactional" or "navigational" in nature. These two seemingly obvious categories are in fact relatively new to the academic field of Information Retrieval (IR), which developed largely in the context of large islands of data (ie, in the 70s/80s), rather than in the web era.

What I like about this paper is the use of the word "intent" - which over the years I've come to use quite a bit (see my last column on video advertising over the internet, in which I rant once again on "intent over content", or my post on The Database of Intentions). Intent is behind every kind of search, Broder says, but "there is no assumption ... that this intent can be inferred with any certitude from the query." Ay, there's the rub....To get to that intent, Broder employed a short survey on the site.

A few fun facts from Broder's analysis of response and related log data:
- nearly 15% of searchers wish for "a good collection of links on a subject" as opposed to "a good document."
- 12% of queries in the log data used were sexual in nature
- nearly 25% of searchers were looking for "a specific website that I already had in mind."
- An estimated 36% of searchers were looking for transactional information - what Broder calls "the intent to perform some web-mediated activity."

Broder concludes that the next generation of search engines will need to take into account this new taxonomy of intent - transactions, navigation, as well as informational. Given that this paper was published in late 2001, it's interesting to see how the major engines already are on that path - with Yahoo's focus on shopping being one of the best examples.

A very kind reader (I'll buy the drinks next time) has forwarded me eBay's recent analyst day presentation (from earlier in the month). This reader's comment: "The focus on search was VERY new...analysts were trying to figure out the significance." Indeed, one of the slides declares: "eBay is Search!"

A couple of things come to mind when reviewing the slides (there were more than 300,covering the entire business, both US and int'l). First, eBay knows that the faster and more relevant they can make their internal search, the better their margins. They understand that they have a marketplace of people with very specific intents, looking for very specific things. The easier eBay can make it for folks to find what they want (whether it's a bid, an item, or a comparison), the better their bottom line looks. Toward this end, eBay has built its own internal search engine called Voyager, which is optimized for eBay users. One of the slides in the analyst presentation boasts: "We are world class at analyzing our user base."

Second and possibly more important is the role of search in customer acquisition. eBay has a world class IT solution in place to monitor tens of thousands of paid keywords across the web, each with its own P&L and analytics. I can't confirm this, but I would not be surprised if eBay is Google's largest customer, something that probably makes both companies uncomfortable, because each can analyze the other's data and mine it for competitive edge. In the presentation eBay also notes the power of natural or algorithmic search (the "pure" results) - the company says it is revamping its entire site to optimize for natural search. Now that's quite a statement. Again, major thanks to my source for this information.

Among the analysts still covering the internet after the great wipe out of 2001, Safa Rashtchy, of Piper Jaffray, has received the most notice as an early and ardent supporter of search. His company will soon come out with a new report forecasting online advertising revenues (Safa is credited with pegging the paid search segment of the market at a widely reported figure of nearly $7 billion by 2007). In his recent newsletter, which summarizes a conference Piper hosted on online advertising, Safa predicts revenues will sharply accelerate, to more than $15 billion by 2008. Also, he predicts that online ad revenues will match 2000's number of $8.1 billion - the height of the boom - by next year. My guess: he's wrong - when the counting's done, 2004 will beat 2000 by a comfortable margin.

I'm offline most of today, but enough folks have passed me these two sites as interesting cultural outgrowths of search that I'll post them here for your review. The first, GoogleRace, plugs into the Google API and ranks each candidate by keyword searches. Some of the tops searches are "large penis" and "George Bush is going to lose". (Thanks Kenny..)

Googlehouse, on the other hand, is more understated. This is an attempt to make a cultural commentary through images - the site polls the Google Image search database and pulls up images that fit various parts of an imaginary house. Click around a bit, if you've got the time. This kind of stuff usually pushes my MEGO button, but...give it a looksee. (Thanks Tim...)

And another note. Thanks to this blog, many readers are now starting to ping me with interesting stuff, tidbits, even insights for my book. I really appreciate and encourage this. I'm at jbat@battellemedia.com, keep those cards and letters coming.

So I printed out three papers suggested by Gary Price in this post. I read the third one first, and didn't find it earth shattering, though there were a few interesting tidbits. The paper is titled: "U.S. Versus European Web Searching Trends" by Amanda Spink and Bernard Jansen (Penn St. Univ) and Seda Ozmutlu & Huseyin C. Ozmutlu (Uludag University). Basic conclusions: US searchers tend to use fewer words in queries, and tended to have shorter search sessions overall. Also, European users tend to look at more query results, compared with US searchers, who were vieweing fewer results per query. (This buttresses the stereotype that US citizens are more impatient and less deliberative than their European counterparts).
Also consistent with stereotype was a comparison of general topic categories searched for by each group. For US searchers, the #1 topic, with nearly 25% of the overall searches, was "Commerce, travel, employment, or economy." That category was # 3 for European searchers, with only 12.3% of the searches. European's #1 category was "People Places and Things." Also, it seems that Europe (recall this was in 2001) was still on a learning curve for tech, as the #2 search category was "Computers or the Internet." That term was #4 for the US during the same period. Also telling: European searchers were more than 4 times more likley to look for for "Performing or Fine Arts" than US users, and not surprisingly, "Sex or Pornography" was two places higher on the European list, coming in at #4.
The study goes on to conclude, though not very forcefully, that there are noticeable differences between US and European searchers, but the authors don't claim it's necessarily a cultural thing, it may well be the distinction in the engines themselves, as much as anything. This study left me wanting more, and happy they have continued this kind of work. (I'll be reviewing this latest find soon.)

Looksmart today announced a major update to FindArticles, a great idea whose execution so far I can't quite endorse. The service - which has its own tab on the Looksmart site - has "over 3.5 million articles from over 700 publications." While I've not explored it fully, a search for various folks who I know are subjects of major magazine pieces turns up a boatload of BusinessWire-type press releases. Not sure how that matches up with expectations of "articles from 700 publications..." but I hope they clean it up, because it could be a great service.

A new study says the Yellow Pages will continue to grow but...there's a catch. I've taken to quoting the current size of the Yellow Pages market - $26 billion or so - as proof there's a lot of growth left in search. After all, if non-local search is about $2-3 billion this year, and is poised to undermine the local Yellow Pages market due to its nascent push into local search, that's a rather large market to grow into. But a study by the Kelsey Group, made public by eMarketer today, says that the Yellow Pages themselves will also grow in the next five years, to $36 billion. However, a full 23% of that 2008 revenue - about $10 billion - will be "digital directory" Yellow Pages - yup, paid local search.
OK, so the math is: Yellow Pages is a $26 billion biz now, growing to $36 billion by 2008, a $10 billion increase. Local Yellow Pages "Digital Directory" is - well not much now (as far as I can tell it's not even in the current $26 billion number), but it will be $10 billion by 2008. Given that the local search piece of the business is quite small today, and the online yellow pages is probably no more than $300 million, if that, what this study really seems to say is that the lion's share of ALL growth in Yellow Pages will be digital. In other words, there will NO growth in the Yellow Pages market, if it weren't for online search. Now that I can buy into.

The Boston Globe also has a "worm turns" piece here, and one of the key sources in the piece takes offense at how his quotes were used...Another source (Dave Winer) also feels wronged... I find this kind of journalism, where reporters try so hard to make quotes fit into a pre-concieved notion of how a story should play, the most irritating thing in business reporting today. In this case, it's not the Google Is Wonderful Look At All The Lava Lamps angle, it's the opposite: Google Is Making Enemies And Is Too Big For Its Britches. Neither is right. Ugh.

Fortune this week proves that the worm can turn in the mainstream media's coverage of all stories, even one that for years has proven my predictions wrong. At least 18 months ago I was cluck-clucking to the communications honchos at Google that they should "beware the backlash." They were getting too much good press, and at some point the media always wakes up and eats its young. But Google enjoyed the longest free ride I've seen in recent history - even when those same PR honchos essentially went dark and refused to give anyone much access. The same story kept getting written, again, and again, and again....
This Fortune piece isn't a hit, but it does rehash the negative bits with at least as much ardor as the positive ones. The piece sets up like this: At Fortune we did some *real* reporting (the implication being that all the stories before were stage managed affairs), and we found out that the company that everyone's been lauding for the past three years is...complicated, contradictory, and not exactly perfect. Not a rocket science conclusion, but it manages to make the company seem a bit more human. The piece states some very old stuff as fresh (that the company has recently grown arrogant - this is new?), has tidbits of news known only to a very few insiders (that Bill Joy and Google flirted but eventually did not come to terms), and a couple off the record sources who are also investors saying stuff like: "Google has a lot of momentum, but its current position is probably not defensible." (Yahoo holds 5% of Google...).
The piece is a fine round up of where things stand, but I can't help feeling a bit empty - so much of the story was stuff we already knew, but had to be in there because of Fortune's large readership, not all of which could be assumed to be avid followers of all things Google. That's the problem with mainstream media coverage - it has to speak to everyone. The story lacked an analysis of business models, of the industry, and any deep discussion of the larger phenomenon Google represents. And the negative bits, while something of a first, had a twinge of gossip and/or sour grapes to them. Overall, I'm not sure this piece moved my view of Google one way or another. How about you all?

I'm starting to read more academic papers, research presented by various professors and the like (including some folks who are technologists working in the industry). One of the things I find fascinating about the search business is how quickly it's turned from an academic pursuit - with all the implications of open, non-commercial sharing of findings - to one driven by clusters of high-powered nerds hitched to a particular corporation's R&D machine. I've been asking around on this idea and found most landlocked geeks agree - Nutch aside, a good percentage of search research has by and large been silo'd - many of the best minds are at Google, Microsoft, Yahoo, and a few others. I'll wager that between the 500 or so top engineers at these corporations, there ain't a hell of a lot of sharing going on. Robust peer review between bare-knuckle competitors? Prolly not.
But it was not always so. Recall that both Yahoo and Google came out of the Stanford CS department, for the most part. Same for Excite (Joe and Graham). Lycos was midwived by CMU, and out of Berkeley came Inktomi. Anyway, you get the picture. A lot of innovation came out of the publish-and-peer-review culture of the university setting, and many of the folks who drove that culture are suited up, so to speak, in one corporate silo or another.
In any case, before they joined up, many of them wrote wonderful academic papers they shared with all in the name of progress (some still do). And there are still plenty of great academic researchers banging away on the database of intentions, though certainly they don't benefit from owning their own slice of it like the majors do (many borrow data from the majors and perform analysis on that). So as I work on the book, I'll be posting reviews of some of the papers I read - the interesting bits, so to speak. I'll title each entry "The Search Papers: Cute Name Here" for ease of use, or more likely, as a clear caveat that discussion of academic Mumbo-J will follow. The first will be out in the next day or so, a wonderfully predictable little study (found via Gary Price, of course) comparing European and US search patterns from 2001 FAST and Excite (pre-Chapter 11) data. Hope you like it.

Here's the 2.0 link, but you have to be a subscriber to get past the first page. So here it is as a pdf file (caveat: it will download when you click). I'm still working on putting my Biz 2.0 columns on this site in a more reader-friendly fashion...This month's points to the idea (not really new now, but it felt more so six weeks ago when I wrote it) that the geeks behind the best innovations on the net are starting to get excited again over interesting problems that have Very Cool solutions, after two or so years of nuclear winter. Boing Boing points to another columnist who's recently seen the same trend.

OK, this is just too much. A major, respected school of business issuing what amounts to a puff piece/PR gloss on Google, quoting professors from various branches of the school (legal, marketing, etc), who manage to say just about nothing new. They don't even answer the question raised by the article's headline ("What is Google Worth?") What on earth are they doing? Why, riding the Google PR machine, of course. "Hey, here's an idea! Let's round up all our professors, have them say smart-sounding things about Google, then take credit for regurgitating old ideas!"
Hurray for bandwagonism. I'd wager the folks at Google are a bit embarrassed by this.
This piece at Cnet shows the school has been paying attention to the company for some time. Come on UPenn - if you are going to pick up the discussion, add to it at least, especially as an academic institution.

I am a subscriber to John Brockman's Edge, which is sometimes (well, for me as a rank average mind, more than sometimes) insufferable for its - well, its intellectual preening (it's basically conversations with folks who are scary smart) and its steadfast unwillingness to make itself reasonably approachable (its last missive, which comes by email, weighed in at 8900 words and seems entirely unedited, for example). But, at times I will plow through the thing, and on a recent occasion, I did just that when along came an interview with none other than Jaron Lanier, Wired staple way back in the age and a constant reminder of how fun and freaky those days were. (more below)

Continue reading "Lanier (yes, Jaron) on Software, Biology, Complexity, and ..." »

Adweek reports Carat Interactive has appointed Ron Belanger vice president of search engine marketing.
Excerpt: "In the newly created role, he will guide the strategic vision of the interactive shop's 10-person search-engine marketing practice, which handles paid placement, paid inclusion and search-engine optimization for clients like Hyatt Hotels. He will report to Carat Interactive evp, managing director Alan Osetek and will be based at the Boston headquarters."
This is one more sign that Search has come of age in the media world. (via MarketingWonk)

Upshot: they don't like the idea much. Nor do I, but it's not because I hold an unfettered libertarian worldview. I thought the meme of "let's regulate Google like a public utility because it has too much power" was a short lived one, and I'm not sure why Cato brought it up again, but, there you have it. The piece has a lot of links in it and therefore provides a pretty good overview of the "Is Google Too Powerful" concept. Plus the piece even quotes one of my columns, though without direct attribution (it does point to my article hosted and mirrored on some other site...hey...wait a minute! But I guess that's one way to get my stuff out there beyond the walls of the Time Inc. enclave....) It's nice to know folks are reading...

Today I was offline most of the day, as I was working on the book. I met with key folks at Yahoo Search and Scott Rafer , the reasonably new CEO of Feedster (Feedster is a search engine which trolls only RSS feeds. What's an RSS Feed? Read this). Lots of discussion about the present, past and future of search. I left Yahoo impressed with where the company is and how the folks I spoke to approached the problem/opportunity of search. Yahoo has the longest history and the richest resource set of any of the current search players, and they certainly have got serious search religion, something that definitely waxed and waned over the past five or so years. Much talk of the lessons of Google, the looming competition with Microsoft, and good discourse on the best ways forward w/r/t search solutions. Ninj Srinivasan, employee #5 and Editor of Yahoo, has been there for nearly 9 years and is still stoked to come to work every day. That says a lot to me. Funny side note: while I was there, 1600 Yahoo employees and various friends went out into the parking lots and broke the world record for simultaneous yodeling. I'm not kidding...

Meanwhile, Scott at Feedster reminded me that PageRank was created back in a time when making links was pretty hard to do, and therefore a scarce resource, so one could reasonably trust that the links were authoritative. But, he supports the idea (as do others, see here and here) that blogs have muddied the waters for PageRank to the point of diminishing returns. I am certainly not one with the chops to judge, but I do wonder how this meme is playing out now that it's been in the world for a while? It's not like anyone has come up with anything demonstrably better....
In a related note, Scott mentioned an intriguing development at Feedster that he calls Feedpaper. From his site: "Feedpaper is a dynamic newspaper constructed from RSS feeds around a particular topic." They are playing around with the idea on the site, for examples, check out this one on the Dean campaign and this one on the recent Longhorn developer conference (Longhorn is the new version of Microsoft's interface, due sometime later this decade...). Scott's thesis is that algorithmic search engines do a pretty poor job of aggregating this kind of content for readers interested in a very specific "island" of information. Watch this space, it could get interesting.

When I was at AdTech way back in June I met with Karen Howe, CEO of Singingfish, an audio/video search engine. I believed then and still believe now that video and audio files will become integral to the grammar of the web, but first we have to solve major search and copyright issues. In any case, I enjoyed meeting her, but left feeling like they had a significant uphill battle - a very small company in a land of giants.

Well, no more. I was messing around on Google news earlier this afternoon and somehow I came across a very odd link from the Puget Sound Business Journal about Singingfish being sold to AOL. Google's summary had details: the excerpt read "has acquired Seattle-based Singingfish Inc., which makes a search engine that scours the Web for audio and video files. Terms of the deal were not disclosed. ... " but when I clicked on the link, it took me to a story with the headline "MagnaDrive raises $3.5 million ". Turns out the Puget Sound Journal had jumped the gun on a embargoed story, and Google News grabbed it before they could swap it out. The news officially breaks tonight at midnight, but ... you saw it here first!

As to what it all means - I find it a bit out of character for AOL to acquire a search engine - and even more so one of this stripe. AOL has formidable serving and personalization technology in house, but it's never really been seen as a leader in search. It outsources that to Google - though it is true that Google does not do multimedia search (Alta Vista and AlltheWeb do). Is this a bid to change all that? Stay tuned, I'll keep you posted.

UPDATE: USA Today has a story on the sale now...

GrokLaw, a blog run by a paralegal who must be hepped up on triple espressos, has posted an interesting journey into the seamy side of paid search. Basically, this fellow noticed a significant discrepancy between Google and MSN's search results, and set out to understand why. This is a great illustration of a theme that is fundamental to Google's mojo: purity of results. MSN comes across as basically sold to the highest bidder, with competitive manipulations on top of that. It's not a pretty picture. When MSFT launches its new search engine, one hopes they will keep this in mind.

In this post on his blog, Dan Gillmor of the SJMN points to the contradiction between "company executives and others" who claimed MSFT approached Google about a buyout (see my earlier post and comments here) and Gates' very clear denial of same earlier this week. Dan points out that Gates is the CEO of a public company, so he can't very well lie about something so material to his stock price. Because of this he implies that Google insiders - such as the VCs who backed the company - are the likely sources of the story.

I'm not so sure. The sourcing in the Times piece seems intentionally non-specific:

"According to company executives and others briefed on the discussions, Microsoft - desperate to capture a slice of the popular and ad-generating search business - approached Google within the last two months to discuss options, including the possibility of a takeover."

Which company does "company executives" modify, MSFT - which is closer to the sourcing, or Google? Who knows?! My guess is the Times kept it vague on purpose, to protect its sources. (more via link below)

Continue reading "The NYT, Yesterday's News, and Plausible Deniability" »

Internet News reports:

"Software giant Microsoft is testing its answer to Google's popular news aggregator and search site. "MSN Newsbot", on MSN UK, France, Spain and Italy, signals at least one of Microsoft's intentions as it seeks to build out its own search technology."

This is one area where MSFT has some serious prior chops - MSNBC has been thinking about news online for a long time. Should be interesting to see how it shakes out in this market.

This one from Search-This, a SEO/SEM house. The buttons, when clicked, activate colored arrows that chart who supplies who with what...ie Inktomi powers MSN, Google powers AOL, etc. (Thanks to Josh Quittner for the reference...)

Once a year Google holds a contest for hardcore search-related coders. This year, the Europeans swept, interestingly. But no mention of what they actually did (as I recall, last year they asked for new features, and the winner made a local search app that Google ended up incorporating into Google Labs)? I'll look around...Aha...Read Slashdot threads here to find out more on the problems the Google coders were solving...MarketingWonk has a round up here of the PR and Marketing implications...

In the past few months I've gotten a fair number of similar email threads forwarded my way by friends who know I'm writing about search. By the time they've gotten to me, the emails have wound their way fairly well through the six-degrees-of-separation web, with scores if not hundreds of souls cc'd, forwarded, and attached. The subject line usually blares something along the lines of "I can't believe they can do this!" and "Oh My God, Did You Know?"

Here's a sample email, with identifying information deleted:

-------------------
Subject: This is hard to believe, but true, I tried it.

Google has implemented a new feature wherein you can type someone's
telephone number into the search bar and hit enter and then you will be
given a map to their house.
Before forwarding this, I tested it by typing my telephone number in
google.com. My phone number came up, and when I clicked on the MapQuest link, it actually mapped out where I live. Quite scary.
Think about it--if a child, single person, ANYONE gives out his/her phone
number, someone can actually now look it up to find out where he/she
lives. The safety issues are obvious, and alarming. This is not a hoax; Mapquest will put a star on your house on your street.
---------------

I understand the initial reaction of many to this feature (which is not that new). My God, They Know Where I Live! But this fear of a such a simple thing - a reverse directory lookup - bears further contemplation. Fact is, reverse directories are not illegal. But they are also not widely available – usually only cops and reporters had access to them. No more. (more via link below)

Continue reading "Why We Wish Some Public Information Would Remain Bound To Atoms (The Public Privacy Dilemma)" »

When Google flutters its wings, a typhoon may result in the AdWords ecology - those hundreds of thousands of advertisers who depend on Google for sales via the company's paid listings. So points out CNet's Stephanie Olsen in a good overview of advertiser reaction to Google's most recent shift in its AdWords technology. The complications Olsen reports point to a larger story: the increasing complexity of this shifting market ecology. The question then becomes, can any one company maintain control of this? I don't think so, you need robust competition; the recent defection of Paul Ryan (former CTO of Overture) to MSN will help insure robust competitors for years to come.

This has been in the works for sometime, but MSFT formally announced its earnings yesterday, and broke out the MSN unit's numbers for the first time. Headline: It's all about paid search. The highlights (from the MSFT IR site): MSN had 50% quarter-to-quarter advertising revenue growth, total revenue totaled $491 million in the first quarter compared to $427 million in the prior year’s first quarter. MSN Subscription revenue declined $17 million or 6% reflecting a decrease in the number of subscribers. MSN Network services revenue grew $81 million or 51% as a result of growth in paid search and strong general advertising sales across all geographic regions.

Ask Chief Steve Berkowitz (caveat: we ran in the same circles at IDG) gives one of his first interviews (to the CC Times) since being formally named CEO. Steve's a good guy and he has quite a job - being #4 in a three-horse race ain't fun. But he lays out his plans and makes his case in the interview.

Samples:

Q: From whom are you gaining market share?

A: There's Google, Microsoft, Yahoo and AOL. I'm sure we're picking up a little from each. We grew our queries by 33 percent in (the third quarter). We believe the market grew at 10 percent. That means we're gaining share. Being a small player, that's pretty significant for us. I don't think the other people are noticing it. Google is focused on Yahoo, and MSN is focused on Google. They're all focused on each other.

and...

Q: (What's next for consumer search...)

A:....The next generation of search is going to be about getting underneath the needs of the user. One of the interesting things we're testing in research is if someone types in the word "AF," do you mean Air Force or do you mean Abercrombie & Fitch? We're developing technology that will get underneath that and realize that most people mean Abercrombie. Or if I know you're searching about upstate New York, and then you type in the word "apple," you're really not looking for Apple, the computer company. I can tell by the five searches you did previously that you're really looking for an apple orchard, so maybe I can skew your results towards that. It's getting into understanding the behavior of both the session you're in, as well as what the masses are saying are better results. It's about integrating all these different things back into the user experience, so what you get back is a much more perceived, personalized result.

So nothing really new in the news today, I wanted to take a graf or two and explain what I mean by The Database of Intentions, referred to in this post. That way I can use it again and again and just link the phrase to this post. Hey, we love the web, Ted Nelson lives....

The Database of Intentions is an idea central to the book I've been working on for the past year or so, which is tentatively titled "The Search: Business and Culture in the Age of Google" (Penguin/Putnam/Portfolio 2004). As with many in this industry, it all started with the Macintosh. Back in the mid 80s I was an undergraduate in Cultural Antropology, and I had a class - taught by the late Jim Deetz,which focused on the idea of material culture - basically, interpreting the artifacts of everyday life. It took the tools of archaeology - usually taught only in the context of civilizations long dead - and merged them with the tools of Cultural Anthropology, which interpreted living cultures. He encouraged us to see all things modified by man as expressions of culture, and therefore as keys to understanding culture itself. I began to see language, writing, and most everyday things in a new light - as reflecting the culture which created them, and fraught with all kinds of intent, contreversies, politics, relationships. It was a way to pick up current culture and hold it in your hand, make sense of it, read it.
(more via link below)

Continue reading "The Database of Intentions" »

It gives me something of a thrill to reference The Standard in an article about booming online ad sales, and give credit to Matt McAllister, who runs Infoworld's site and took over thestandard.com as a sidelight, as IDG was about to shut it down along with all of IDG.net. The archive is still not up, but Matt vows it will be, and the stories are all headline retreads from other IDG publications, but, there's still a pulse there. Also, it's really poignant to see contextual ads on the site, after all the dreaming I did of CRM doing - far too expensively - what contextual ads essentially can do now. Thanks Matt!

Via MarketingWonk, I saw this short blurb in Crain's NY business which points to Barry Diller's increasing show of muscle in the local search market. He's got Citysearch and various other localized online businesses, and Yahoo, MSN and Google are all hot for the opportunity to extend their advertising networks into the local market. Diller seemed to be at his Cheshire'd best on an earnings call Tuesday.

Eye opening Salon piece on Universities' practice of logging the net usage of their student populations. It notes that these practices are under review as the Patriot Act and RIAA are subpoenaing the logs, which many universities kept as a matter of course (why? who knows). This is another example of the power of the Database of Intentions (a term that is central to my book, and I promise, I'll explain at some point). Interestingly, the Patriot Act may well be responsible for the widespread loss of this valuable resource (see excerpts).

Excerpts: "At the University of California at Berkeley, the everyday Web-surfing habits of students are regularly watched and recorded. Berkeley's Systems and Network Security group uses a program called BRO -- named after the infamous fascist icon from George Orwell's "1984" -- that keeps logs of every IP address students visit on the Internet from the campus network.

Cliff Frost, UC-Berkeley's director of communication and network services, says that "this practice is under review right now," because the campus community feels it interferes with academic freedom. He expects that the university will continue to keep logs but will discard them after a month or two. "I'd love to keep that data forever," he adds, "if there weren't the threats of subpoenas for vile purposes." "

And...

" The only way to defend student privacy against USA-PATRIOT subpoenas, says University of Michigan public policy professor Virginia Rezmierski, is for university IT departments to stop saving their logs. You can't subpoena information that doesn't exist. Rezmierski is the lead author of a 2001 National Science Foundation study of network monitoring and logging practices on college campuses.

"I don't think this study made people very happy when it came out," she says. "A lot of our findings were very disturbing." She describes interviewing a college systems administrator for the study who told her that he had singled out one student and periodically logged everything he did on his computer "because [the student] was really competent with network operations and he seemed a suspicious type." "

If I ever put in AdSense, I am sure I'll get detox ads, as this is my second drug-related post, but...interesting to see yesterday that various drug companies and outlets are pressuring search engines to police advertisements for illegal pharmacies and/or drugs. This is a huge business, and it makes me wonder, are they also asking ISPs to police spam? Because I get a shitload of offers for prescription drugs. Just who is responsible here? Combined with trademark issues here and abroad (eBay has asked Google to stop using "eBay" in keyword advertisements), this is the beginning of something quite interesting.

eMarketer has a stat-filled story today outlining advertising spending by small to medium-sized enterprises (SME). It's quite heartening - if you add up website, email marketing, keyword search and banners, online ad spending is a strong second place in overall ad spend for this category, behind only the Yellow Pages. According to the study, done by The Kelsey Group, online will capture 16% of a $22 billion spend, or $3.52 billion. The study goes on to say, however, that online will average only 2% of the total US advertising spend of $230 billion, or $4.6 billion (larger advertisers are only adding in another billion? Huh?!). Everyone counts differently on this metric, eMarketer says online ad spending will grow from $6 billion last year to $6.9 billion in 2003.

Steven Johnson , founder of Feed and author of Emergence and Interface Culture, also writes a blog about this, that and the other thing. He recently added Google's AdSense contextual ads , a program I've applauded as darn near revolutionary in its ability to support micropublishing. But Steven has run into a problem. In his own words:

"Jesus, one lousy post about Rush Limbaugh and Courtney Love and every single GoogleAd on the front door is for heroin detox programs. Kind of a downer, no?
(Of course, by adding a new post with the phrase "heroin detox program" I've just made matters worse. Oops, did it again.)"

(For more, here's the permalink.)

Steven also points to the ongoing problem of comment spam, which I've also been the victim of, even though this site is only a week or so old. F*ckers.

Clay Shirky, a prolific writer/thinker on subjects net-related, has made a strong argument against the "Semantic Web." Why should you care? Well, the Sematic Web is Tim Berners Lee's vision of the next version of the Web, a rather seductive vision which addresses many current shortcomings. And since he invented the first version, it gets some serious notice. But Shirky points out, in a very readable and convincing fashion, why the whole idea simply won't work. Some excerpts: "After 50 years of work, the performance of machines designed to think about the world the way humans do has remained, to put it politely, sub-optimal. The Semantic Web sets out to address this by reversing the problem. Since it's hard to make machines think about the world, the new goal is to describe the world in ways that are easy for machines to think about." "There is a list of technologies that are actually political philosophy masquerading as code, a list that includes Xanadu, Freenet, and now the Semantic Web. The Semantic Web's philosophical argument -- the world should make more sense than it does -- is hard to argue with. The Semantic Web, with its neat ontologies and its syllogistic logic, is a nice vision. However, like many visions that project future benefits but ignore present costs, it requires too much coordination and too much energy to effect in the real world, where deductive logic is less effective and shared worldview is harder to create than we often want to admit. "

The Las Vegas airport will soon implement RFID tags on all bags sorted through its facility (about 65K-70K a day). Now RFID tags - RFID stands for Radio Frequency Identification - are "small integrated circuits connected to an antenna, which can respond to an interrogating RF signal with simple identifying information, or with more complex signals depending on the size of the IC." (source) In other words, they are tiny little units of searchable information attached to your luggage, telling airport officials - and we hope only airport officials - who owns the luggage, whether it's cleared security, etc. But as RFID spreads - and it will (Walmart is installing the technology to track inventory) - it brings an entirely new dimension to the idea of what it attached to the network, and what can be searched beyond the web. Perhaps if you lose your luggage, you'll someday be able to find it via Google....

If you're a Windows user (and I am not, so this is not a firsthand look), you might check out the new Google Deskbar, which came out this past Thursday. It's an interesting bid to move beyond the browser and challenge MSFT directly at the Windows level. Definitely watch this space. From the Google FAQ : "The Google Deskbar is a Windows application that lets you search using Google anytime your computer is on and connected to the Internet -- even when your browser isn't running. Google startup search puts a search box in the Windows taskbar at the bottom of your screen and displays your search results in a small pane that rises above it. It provides instant access to information on the web, no matter what application you're using."

5 exabytes, according to UC Berkeley's SIMS (School of Information Management). If you are into statistics and love media, this study is a real gem. Read the executive summary to find out all sorts of cool stuff, like the fact that all worldwide telephone traffic in one year creates 17+ exabytes of data (and exabyte is 1,000,000,000,000,000,000 bytes OR 10 18 bytes - or - put another way, 5 exabytes is all the words ever spoken by human beings.). Alrighty then.

Google and the public's lack of understanding of what libraries have to offer is killing public demand for the librarian's skills (and by extension, dumbing down the world),writes Gary Price, a thoughtful (and arguably endangered) information science professional. (Gary maintains ResourceShelf, considered by many to be one of the best sites for his profession). "What happened?," he asks. "How have we gotten to this point? Why do more and more people believe that universal truth is just a click away via a single source?" My answer: because it seems so, and for most, that's enough...(thanks to Rex Hammock's blog for pointing this out).

Yahoo made some minor changes to its home page this week, but as CNET reports, minor changes to Yahoo's home page are a very big deal. The changes emphasize Search, of course.... From the piece (this ain't news, Yahoo's been focused on search for sometime, but it's evidence the focus is bearing fruit): "The design changes reflect an internal mandate at the company to make search a prominent feature across the entire network, including Yahoo's personals, travel and real estate sections. The impetus is largely fueled by the shifting economics of search and its lucrative advertising component, pay-per-click sponsored listings. "

I'm going to try to avoid blogging everything Danny Sullivan says, but he says it so well, so often. If you're trying to make sense of the last week's headlines about Google, MSFT, and the IPO, read this.

Reading the TV news this Fall, it's hard not to believe the sky is falling, or at least several iridium-laced meteors have hit the networks, with more on the way. It's like watching an entire ecosystem fail in real time: Networks cancel major shows after two or three episodes, TV execs wring their hands at terrible ratings losses ("Where Are The Young Men?"), finally declaring, after trying to lay the blame at Neilsen's feet, that it must be a programming issue. WELL OF COURSE IT'S A PROGRAMMING ISSUE! Jesus! Who are these people? The programs not only suck, they are completely out of touch with how this generation of viewers - better to say media consumers - want to interact with their entertainment. They don't want to sit at some huge communal campfire, watching the same crap as 20 million other people (especially the young men!) And yet, the networks continue to tart up old formats like the sitcom or reality TV (yes, it's tired too), then sit back agahst when they don't pull in Superbowl numbers. They lurch after every possible "hot" idea that worked on cable or with a smaller audience (Gays! Reality TV! Buffy!), then try to force it into the mass media superstructure. Memo to the networks: there's a reason shows like Buffy, or The Daily Show, or Queer Eye have non-network numbers - they're NOT NETWORK SHOWS. They have natural audiences that do not scale to expectations of Thursday night primetime. It's amazing to see a show like Coupling, which had a cult following in Britain, being canceled because it only pulled an 8.1 - my god, that's huge by any standard other than network. Networks are slow to realize it, but the days of Big Audiences With Nothing In Common are over, save possibly news, awards shows, and sports. And good riddance. The pap these networks are putting on the air in an attempt to not offend anyone is getting simply indefensible .

A fun chart from a SEO site called "IHelpYou, Inc." outlines the major partnerships and relationships between search companies. It's not complete, but it's a good start. (Thanks to the GooGuide for this.)

The eBay-and-Google-are-competitors meme continues unabated, and Meg Whitman responded at a recent analyst day, "We're good for Google, and Google is good for our sellers." Ebay spends buckets on Google's AdWords program, and it also spends buckets over at Overture, which recently got some good news from Jupiter, which published a study claiming Overture's paid listings service outperformed Google's.

Ho hum. Not as sharp as usual, but a few good points in this Economist piece on the Google IPO. Chief among them is a claim that Google's contextual advertising margins are weak, according to Findwhat's CEO (wishful thinking, perhaps?), and a furtherance of the building meme that Google is, in fact, an online advertising agency driven by a great search engine.

Worthy of a read, and related to my earlier post about architecture and politics, Declan at CNET outlines current administration tactics w/r/t information sharing that taken together certainly seem to form a pattern. We should all be thankful for projects like Brewster's Internet Archive and The Memory Hole. UPDATE: Larry Lessig points out that in fact the White House has deliberately changed the press release, and the Internet Archive proves it. No record made of this by the administration. Thank God for the IA.

Dipsie, a new search engine coming Summer 2004, was recently written up in Businessweek (two paragraphs, with no mention of who was behind it). I could not find the site using a Google search (or Alta Vista or Teoma, so hold the conspiracy theories), but found it by simply typing in "dipsie.com." Gee...that seems odd, or they are so new that they've not been crawled, or...they have banned spiders from their site. The site is very light on details, though it does say they are privately funded, in Chicago, and they are hiring. In the Businessweek piece (I'd link to it but their online site is deeply f*cked up) they also claim they will index 10 billion web pages (Google et al do about 3 billion) and will have semantic ranking (language based) that will be better than Google (the motto is "Get Where You Want In Two Clicks".) If that's right, expect them to be bought soon. If they *really do* have great tech, tech that can scale and withstand serious slashdotting, they'll lash themselves to the mast, ignore the siren call of easy money from Google, MSFT, or Yahoo, and launch. Their price will go up with every new user they pull in. As Eric at Google often points out, the competition is one click away. And as great as Google is, fact is, no one ever gets past what Tim Bray calls the "Google Event Horizon" - result # 1001. In fact, few get past result #10 in any search engine, and therein lies an opportunity.