John Battelle's Searchblog

Not that I think most will want this kind of dashboard, but it's a neat hack - Scoble points us to lushe.net, which forces Google to search only the sites you tell it to.

This site is a project by Glen Murphy. Who is he? I have no idea, but here's his site. This is why I love the web.

Google has a page up reacting to the Gmail privacy brouhaha. Thank the California Legislature for this: a modified version of the Gmail privacy bill just passed the CA Senate.

Since the first privacy concerns were raised nearly two months ago, one of the smartest things Google has done in response is to give accounts to a bunch of journalists (I was on the list, but have not really used the account - I'm too swamped to conjur up a reason to fill a secondary account, and in any case I'm writing a book in which Google plays a critical role, so the idea of running my email through their servers feels a bit...odd, to say the least. I can just see my sources at Yahoo or Microsoft wondering whether I'm capable of fairness as they send mail to jbattelle@gmail.com...).

Anyway, those journalists who did use the service almost universally praised it. As a breed journalists are prone to piling on when they identify what they believe to be a clever or counter-intuitive meme. Gmail offered them a pretty prime opportunity to do just that. Most caught the counter-spin on Gmail's scanning of email - to wit, everyone else does it (Hotmail, Yahoo etc) so why jump on Google? They are right, and it was a brilliant move on Google's part to point that out. Now the press is full of articles beating up the privacy advocates, and as I mentioned earlier the California legislator who introduced the grandstanding Gmail privacy bill has modified it to allow scanning.

Google's page points to all these journalist's reviews as proof that the whole issue has been misconstrued, and in some regards they're right. But I think the debate is light on a very important, larger point. It's not that Gmail scans your email, or that it might take a while for duplicates of your mails to be deleted off the system. It's that with a gigabyte of storage, user habits with regard to email change entirely, and we start to keep our entire computing life online, moving massive amounts of personal and private communications and files into a new realm of "discoverability" - from the ephemeral to the eternal.

Very much to their credit, Google addresses this issue, and acknowledges their role as market leader requires they be held to "a higher standard." But it's buried down at the bottom, and it doesn't take the logical next step, which is to call directly for clarification around providing email and files stored on a third party servers the same legal status that they already enjoy on our private machines and as they are transmitted over networks, thanks to the Electronic Communications Privacy Act.

In any case, here's the language that I think stands Google in good stead on this issue:

Let's be clear: there are issues with email privacy, and these issues are common to all email providers. The main issue is that the contents of your messages are stored on mailservers for some period of time; there is always a danger that these messages can be obtained and used for purposes that may harm you. There exists a real opportunity for misuse of your information by governments, as well as by your email provider. Careful consideration of the relevant issues, close scrutiny of email providers' practices and policies, and suitable vigilance and enforcement of appropriate legislation are the best defenses against misuse of your information. The only alternative is to avoid new technology altogether, and forego the benefits it provides.

Gmail is still in a limited test period. While we're working to improve Gmail and make it more widely available, we welcome your feedback and suggestions on the service and its features. We encourage users and interested groups around the world to share their thoughts on our policies and procedures by writing to us at privacymatters@google.com.

Net net: a good move on Google's part, and a sign the company recognizes some of the larger issues at play.

- Overview from MediaPost here.

WebSideStory, which first filed four years ago right before the bubble burst, has filed again. The web analytics company just hit profitability after losing money the past few years. It's a great field, but to be honest, this feels a bit rushed to me.

Once again, a good piece from Stefanie Olsen, this time on audio and search. Recall how I am always on about how search will become the interface to media? From her piece today on NPR's decision to optimize their sound files for search engines:

The stakes are enormous, not just for the search engines, but for content owners hoping to harness the Internet, stand out in the online information glut and attract new audiences. The winning search companies could become the gatekeepers in a new era of media increasingly defined by consumers' ability to seek out programming on their own terms and consume when and how they want.

Featured companies include SingingFish and StreamSage.

At the Goldman Sachs internet conference today in Las Vegas, MSN search chief Yusuf Mehdi let slip some of MSFT's plans with regard to search. How do I know he let slip? Well, far as I knew, this stuff was not to be disclosed for quite a while (recall that I was briefed recently by MSFT, on background, for my book).

The news: Microsoft will be including a pretty significant local PC search function as part of its upcoming beta. Mehdi also mentioned that personalized search will be a significant focus going forward, and that MSFT is looking at integrating ads into mail a la Gmail.

From the AP story (Link to an edited version):

SEATTLE (AP) - Microsoft Corp. will soon release technology that takes search functions far beyond the Internet, allowing users to pour through e-mails, personal computers and even big databases to find the information they want, a top executive said Wednesday.

Mehdi said Microsoft plans to release an early version of the technology soon, as part of the software giant's push to compete with Internet search leader Google Inc. A final version is expected in the next 12 months, he said.

During the AdTech panel yesterday I started ranting about what I think is missing from all this contextual, behavioral, paid search, and network-based advertising - you know, all the stuff that's setting records and revolutionizing marketing. All the stuff I've been hyping for the past two, no wait, ten years now. And I think I've come up with a clear way of saying it: what's missing is the advertiser's endemic relationship with the community a publisher serves.

I'm almost certainly restating what others have already pointed out, but then again, I've not seen it put this way yet. So think about a "traditional" publishing environment. You've got three parties in an ongoing, intentional conversation: The reader/viewer (we'll say audience for lack of a better word), the editor/programmer/author/creator (we'll say publisher for lack of a better word), and the advertiser. In a traditional publication, these three parties interact in various ways through the medium of the publication. Most importantly, the advertiser has voted with their dollars for that particular publisher, hopefully because the advertiser had take the time to understand that publication's audience, and hence wants to be in conversation with that audience.

What's inherent in this interaction is the intention of all parties to be in relationship with each other. This creates and fosters a sense of community - the best publications always have what are called "endemic" advertisers - those that "belong" to the publication's community, that "fit" with the publication's voice and point of view. I've found that in the magazines and sites I've helped create, my readers enjoyed the ads nearly as much as the editorial, because the ads served them, seemed to understand who they were in relation to the community the publication created.

It's this relationship which I find entirely missing in all these contextual, behavioral, paid search networks. Sure, they are "relevant" to either a search, or to the content they match. But they are driven by metadata and the actions of only one of the parties - the content of the publisher for example (AdSense), or the actions of the audience (Claria, Revenue Science, Tacoda, etc.). As far as I know, none are driven by an understanding of the give-and-take that occurs between all three parties in a consensual relationship mediated by the publication. A site which has only AdSense or behavioral advertising fails to value (or monetize) the community connection between audience, publisher, and advertiser. Advertisers in these networks are not intentionally supporting the publication, and by extension they are not supporting the community the publication has created. In essence, they are not being good citizens of the community where their advertising is being displayed.

To summarize: Something is lost when advertisers don't buy based on the publication. I'm not arguing that buying based on context or content isn't valuable, it certainly is. But in the long run, not considering the publisher's role devalues both the publication *and* the advertiser in the minds of the publishers' audience.

So what, you might be saying. Most major publications utilize both network-based and more traditional "display" advertising - look at the NYT or CNET or CBS Marketwatch. True enough - Martin mentioned yesterday that his "display" advertising at NYT.com is up dramatically and starting to show real traction. (And, by they way, the NYT is steering clear of AdSense image, for obvious reasons....) But the real problem is with smaller sites, sites that can't afford to be understood or purchased any other way but through a network. Sites where there is simply too much transactional friction to make the advertising purchase worthwhile. Sites like....blogs, for example.

Advertisers can't grok all the blogs which might be potential fits for their marketing dollar. Besides the tedium of finding and evaluating them, blogs have no standardized marketing or advertising practices, so working with each is a handrolled labor of love (I am in the process of managing such a Cohiba right now over at boing boing).

Yet more than most publications, blogs are about the relationship between author and community. Any advertisers who comes into that conversation with a tin ear does a disservice to all parties. My conclusion: If advertisers are going to truly benefit from marketing on blogs, they will have to get to know each one to the point that their ads speak with a voice that is consistent in the community where they are advertising. You can't do that on contextual or behavioral or PPC networks as they exist today.

To its credit, AdSense has given marketers a taste of the blog world, and bloggers a taste of a credible business model. And AdSense spares marketers the trouble of negotiating with each and every site owner. But if advertisers want to find those sites that, in the long term, are endemic to their business, sites which reflect a conversation they'd like to join, they are going to have to actually join the conversation. And we, as bloggers, are going to have to help them join. Henry at BlogAds is working on one solution to this, and I am sure there are more to come.

Over time, I believe advertisers will see the value of long-term relationships with endemic communities of interests. Ideally, they will be able to both buy a specific site, as well as reap the benefits of contextual/behavioral/PPC models. Wouldn't it be cool if we helped them figure this out?

TITANS OF TECH
The Return of the Queen
Morgan Stanley's Mary Meeker survived the boom and the bust. Now she's back -- as are her top stock picks.

By John Battelle, June 2004 Issue

Hailed as the "queen of the Net" by Wall Street tout sheet Barron's in 1998, held up as the poster child of the dotcom bust in 2001, Morgan Stanley (MWD) analyst Mary Meeker has lived through several lifetimes of glory and infamy. She's seen fellow analysts and a former boss indicted for fraud, and then was swept up herself in a massive investigation of Wall Street's research practices (the case was settled last year, with Meeker cleared). What kept her going through all this? The thing most people still don't get is that Mary Meeker is a true believer -- in the companies she helped take public, in the stocks she picked (and stuck with even as they nose-dived), and in the revolutionary nature of the Internet.

Her record is admittedly mixed, with flubs like her recommendation of AOL, which lost $150 billion in market cap after its merger with Time Warner (TWX) (the corporate parent of Business 2.0), and ExciteAtHome, which went from $35 billion to nothing. But in 2003, Meeker's picks were up 78 percent, thanks to stocks she'd long championed, like Amazon (AMZN), eBay (EBAY), and Yahoo (YHOO). If you'd had the fortitude to pile on in early 2001, when she reiterated her support of those companies, you'd be a damn sight richer today. And Meeker is still helping create new industries: Her prescient reports on the search market were part of why Morgan Stanley won a mandate in April to lead Google's IPO. She won't admit to more than feelings of satisfaction with her recent record, but one can sense a certain bounce in her step these days. We visited her in New York to hear what she has learned from the past and what she thinks about the future.
(full text in extended entry...)

Continue reading "June Column: Mary Meeker" »

Take a look at this deal between Yahoo and Plaxo. (Who's on the Board of Plaxo? Yup, Ex-Yahoo CEO Koogle.)
From the the Merc:

Today Plaxo integrates Yahoo's search engine directly into the Outlook e-mail program. Under the deal with Yahoo, Plaxo will get paid for channeling people to Yahoo's search engine. The search box will be placed beside a Plaxo icon that sits atop Outlook.

The deal is significant because it puts Yahoo search directly into one of the most popular e-mail programs. That helps Yahoo leapfrog rival Google, which earlier put its search engine on the bottom of the computer desktop with its Google task bar.

Do you think MSFT is going to let this happen, on *their* desktop, in *their* application, for very long?

(By the way, Google investor and board member Ram Shirram is also an investor in Plaxo...what tangled webs we weave...)

I'll be moderating a panel on "Advertising's Horizons" today at AdTech. Just in time for the event's opening day this news comes out: Online ad revs hit $2.3 billion last quarter, according to the IAB and PwC. Interesting that Q1 04 beat Q4 03, the Xmas season. Now that's strong growth. I still think this market is undercounted - but that's just a gut feel.

If anyone is coming to AdTech and wants to grab a bit of time after, come on up at the end of our session....

Release here....

The A9 chief gave the keynote at the WWW conference, where search played a starring role:

"A9's mandate is to build new search technologies to improve the user experience. We want to invent new things and new ways of finding relevant information. The first question I get from people is, 'Are you going to build another Google?' But, no, that's not what we are doing. There's so much room for innovation that you can build interesting things that aren't available today."...

..."For most users, they expect it to be as simple as possible and that's a barrier. If music was invented 20 years ago, we'd all be playing one-string instruments," he said, suggesting that user habits needed to change to adapt to the advancement in search technologies.

Thanks to Gary for pointing me to these numbers: Census bureau stats on ecommerce growth. Nice up and to the right lines, with good Q4 seasonality. Bottom line: +28% year to year.

I'm late to this game, which means I get to play Monday morning quarterback. As most of you probably know by now, Google has posted a set of principles for software applications here. The title is interesting: "Feedback requested: A proposal to help fight deceptive Internet software."

The company asks you send feedback to an email account. There is a cross post on the Google blog, but comments are not turned on. (Imagine the trolls and spamming they'd have to fight if they did enable comments, sigh, too bad...)

The Slashdot hive mind seems to generally approve of this move. Elsewhere, comment has been mainly along the lines of "well done." Another swish for Google in the White Hat playoffs. And I agree. The principles are very clear, I concur with them on first read. Users have the right to know what software is on their computers, what role it plays in their computing ecosystem, how to get rid of it if they want to, and if and when their experience is being manipulated. All that is good and true.

A secondary line of comments note that these principles may well be covering fire for the introduction of what Markoff reported to be "Puffin," a PC search application expected shortly from Google. Others have noted that they are a shot across Microsoft's bow, and again I concur. But they are also a shot across any number of other players' bows, in this increasingly complicated landscape where partners and customers can also be competitors. If Google were to be truly audacious, these principles just might be a way of undermining competitive ad networks such as Yahoo, Claria, and others.

Why? First, they win by comparison. Google is in the advertising business, and by taking the high road (as they always have in the past), they are laying groundwork for branding any number of competitors as black hats. Think of Yahoo's paid inclusion program. It doesn't take much to lay these principles against that practice, and draw a comparison - Yahoo doesn't tell you which listings you see are paid, and which are organic. That violates the principle's spirit, if not its letter.

Or take Microsoft's bundling of what might otherwise be free standing web applications: if consumers can't "unbundle" or delete those applications, that also might be seen as violating the principles. Sure, the context is different, but high ground is high ground.

But the real juice is in the last section of the new principles, called "Keeping Good Company." It basically says: "We will not partner with anyone who violates these principles, and you shouldn't either." The logic folds upon itself: If you partner with those who violate these principles, you yourself violate them, and we will not partner with you. Now that can get quite interesting.

This is why I can't help but feeling that something else is up here. This sentence was put in italics, and emphasized by Google (under the "Keeping Good Company" section):

Application providers should not allow their products to be bundled with applications that do not meet these guidelines.

It continues (itals mine):

...Getting paid to distribute, or paying money to be distributed with undesirable software enables more undesirable software. Responsible software makers and advertisers can work to prevent such distribution by avoiding these types of business relationships, even if they are through intermediaries.....We believe that it is in our users' and the industry's interest to work to eliminate this problem. For this reason, we will strive to distribute our software only in bundles where all applications meet the above guidelines, and we think users will benefit if others in our industry do the same.

Google partners with a lot of folks, and while I can't put my finger on it, something tells me that this announcement is putting some of those partners on notice in a more broad fashion. After all, what *is* a partner? Someone who distributes Google's services? Amazon, Ask, AOL, About...all partners. Earthlink, BellSouth, the New York Times...the list goes on and on. And most if not all of the ecommerce world uses intermediaries who bundle software with Kazaa like companies. But what if it's also folks who advertise *with* Google? Now that would be interesting. Might Google refuse business from someone who also buys ads on Claria, or paid inclusion on Yahoo? No way, you might say. But then again...

Regardless, I imagine these principles have anyone who has a relationship with Google scurrying to their lawyer's offices, reviewing their own practices, and the practices of their partners, to see if they are violating this new set of terms. (It should be noted that these principles are labeled as "Proposed," so they can be amended should business partners come up with reasonable objections. But I have pretty good sources telling me Google has very carefully considered the document. It's not a beta.)

Also not clear are how some other key terms should be read. In the world of the internet, for example, what's a bundle, and what is "distribution"? Is it just downloaded software? Applications in a box at Circuit City? Co-managed web applications bundled in one place on one site? An ongoing business relationship with AdSense or Overture or Claria?

Similarly, what is meant by "applications"? Just standalone stuff like Toolbar, Deskbar, and the rumored Puffin? Or might this extend to web applications like search and AdSense?

I think it is in fact meant to be read broadly, because the document has this disclaimer at the top:

These guidelines are, by necessity, broad. Software creation and distribution are complex and the technology is continuously evolving. As a result, some useful applications may not comply entirely with these principles and some deceptive practices may not be addressed here. This document is only a start, and focuses on the areas of Internet software and advertising. These guidelines need to be continually updated to keep pace with ever-changing technology.

I wish I could at once see 64 moves down the chessboard in all directions, and predict what the implications might be if these principles were interpreted more broadly. No matter what, many in the industry *will* read these broadly, if only implicitly. The ecology will shift. Google is throwing some weight around here, to good ends, but also to its own. Worth watching. What do you all think?

You know how I get about the concept of the ephemeral to the eternal. From what I can tell, it bores the pants off most of you. But you responded to my write up of Furl, which helps you create a PersonalWeb, and A9, which creates a search history (and more). Given that I am supposed to be writing my book, I won't give you a detailed report on Seruku, but thanks to Gary Price, I don't have to, as he's done it for you. The cool thing about this application is that it allows you to create a local archive of *every page you view on the web.* Now, the fellow behind Furl is quick to point out that Seruku has no comments, no topics, no keywords, and a proprietary format (not to mention, a price tag and no thin client). But still, Gary loves it. And I love that it makes a version of your PersonalWeb that YOU control (trust me, Furl is working on this, and...MSFT, Yahoo and Google are paying attention.) Check it out.

Oh, and by the way. Thank you for all the aid on comment spam. A post on that is forthcoming.

You've heard me raise this issue before, during the initial brouhaha over Yahoo's announcement of paid search, which I tagged as something of a major PR bungle, coming as it did when the world was focused on search thanks to the SES show. Yesterday Danny Sullivan came out with a fine overview of the issue in his SEW newsletter. Watch this space. It is going to get more interesting, that much can be surmised.

Markoff breaks the news officially that Google will have a PC/hard drive search. I was meeting with MSFT folks today when this broke (on my way out from NYC...). That's a story for another day.

Some have asked what the business model is for local hard drive search. My answer: Good will, and lots of it.

The comment spam is overwhelming. I can't keep up. I'll deal with it when I get back from NYC. If anyone has a real solution, I am ready to listen. Now the spammers are changing URLs for every comment. FUCKERS.

Meanwhile, the dinner tonight was quite interesting. A room full of folks in the banking business, buy side analysts, major investment houses, etc. I asked the room - who planned to put in a bid on the Google IPO? Wall Street answered - not a single hand went up. Hmmm.

The Register breaks this "news" - groups of folks get together at Google to determine what to do when difficult issues like the Jew Watch controversy break. Well of course they do. At least the company has an ethics committee in the first place. It's true, they have not been forthcoming on this subject, but as the reality of being public company sets in, they've get better at it. It's an engineering-driven culture in the process of realizing that it's playing on a major media stage. DNA changes slowly, and not without pain.

Would love to say that I've got loads of time to hang with my NYC buddies, but I'm in for a quick dinner, then back to LA to tend to some family issues. But the quick turnaround means posting will probably be light, if that, for the next two days. And, more likely than not, comment spam will once again wend, borg-like, through the lattices of my site. I'll try to manage it from the road....

As Stefanie points out, just about zero, in the new Adsense image scheme. Good point. I sympathize with this quote:

"Google's making a public statement that the only value of a banner is when it's clicked upon, and it flies in the face of all the research done in the last five years that demonstrates the impact a banner can have on brand awareness and purchase intent," said Dave Moore, CEO of 24/7 Real Media, a New York-based company that sells advertising for 800 sites worldwide.

"Why shouldn't I get paid for creating the step to the ultimate purchase?" Moore said.

Brand advertising *is* important, and does deserve some value. Will be interesting to see how this plays out. Clearly, if I am an advertiser and believe in brand advertising, AdSense images make a lot of sense, because (for now) I am getting a lot of free brand advertising. On the other hand, at some point either Google will begin to charge a fee for placement, and/or only image ads which are clicked on will run on the system.

Interesting conclusions by a govt. sponsored panel created as a political bone thrown when the TIA news broke, convened to look into Pentagon and govt. practices.

Excerpts from the NYT coverage:

...The report, expected to be issued in about two weeks, says privacy laws lag far behind advances in information and communications technology....

Mr. Rumsfeld appointed the panel in February 2003 to quell a political uproar over the Pentagon data mining program, headed by John M. Poindexter, a retired rear admiral. Congress cut off money for the program in September 2003, with certain exceptions described in a "classified annex" to the 2004 military spending law.

I have to turn off comments for a while, I am afraid.I can't watch the site every hour of the day, and the comment spammers are winning. I've even got trackback spam. MT Blacklist is not working for some reason, and I've got to figure it out. Sorry about that. But I suggest that if you have comments, perhaps do what Dave suggests, assuming you've got a blog - ping me. The trackbacks will show through (and there is less of that kind of spam), and if enough do, then I'll enable the same kind of approach as boing boing takes.

UPDATE: Got MT working again. It was choking on the amount of incoming spam. Will keep comments open till...next time.

Gary Price noticed that when you search for "behead berg" Google brings up a sponsored link for someone who is selling the video (it didn't come up for me, but I imagine there is only so much inventory the guy wanted to buy). It feels just like a porn site. It's a sign of how we are not quite sure how we feel about watching this video that Google has not put this site on its "bad" list, alongside the t-shirt vendors and the environmentalists (the porn sites *are not* on the bad list...nor do I think they should be...).

Danny Sullivan today writes something of an obit for the brief, glorious phase of Internet industry development in which search played a starring role. Danny has always warned that Google was on track to become a portal in the classic sense, but I think the recent AdSense news, in which Google essentially proclaimed itself a massively parallel ad agency, and the Google Groups news, where Google is playing catch up with Yahoo, has pushed him over the edge. Yahoo's comments yesterday during an analyst meeting didn't help - the company stressed that it was "much more than a search engine" - this after a year in which it focused almost entirely on earning cred in the search field.

Becoming a media player did make great sense for Google. It was a natural move to leverage the large advertising base it has. But its role in placing ads on sites across the web has nothing -- absolutely nothing -- to do with organizing information. ....

..."More than a search engine." It's almost unbelievable to hear those words spoken, especially from Yahoo, which over the past year has been desperately trying to resurrect its image as a search engine. While I've yet to hear Google utter those exact words, its actions speak them loudly.

Is this the end of an era? Yes and no. Yes, in that Google is clearly no longer just a search company, and Danny is right to declare this fact. Google is bigger now, and it has to act like a bigger company. Along with Yahoo, it is one of the first truly "new media" companies of our era. Search is its core editorial product, the internet is its distribution network, and advertising its revenue stream. What's new is that the company breaks some pretty sacred media company norms - distributing advertising across editorial sites that it doesn't control, for example, or abstracting traditional editorial judgment behind opaque curtains of algorithmic logic. Just wait until Google starts to distribute video ads attached to net-based television programming. Don't think it will happen? I'll bet you dinner it will.

But this is not the end of search as we know it, for search will continue to be the engine driving this industry, as the Yahoo folks took pains to point out during their analyst day.

Still, it is the growth potential of search that is receiving the greatest focus at Yahoo these days. The company has a two-pronged strategy for going after the local market that it plans to introduce soon: a primary emphasis on major retailers and other large businesses in a region, and the creation of a locator page for small businesses that do not have any online presence.

"As fast as search is growing, there is an enormous opportunity ahead of us," Weiner said. "As good as all of us believe search results are, we are only delivering high-quality results about half the time as an industry. We have only scratched the surface. We are going to get much better at that."

Should either Google or Yahoo take their eye off the ball again, there are plenty of entrepreneurs ready to take advantage of their missteps. Search has taken its place as an enabling application to the most important new medium in the history of media. And that's a pretty big deal.

You come across some funny shit doing research on the web.

Florence, Italy - Historians have found convincing evidence that Leonardo da Vinci developed the first search engine. Recently uncovered Da Vinci diaries describe how he collaborated with his neighbor, Googlio, to enable a renaissance in searching....

Googlio's popular searches were profitable for da Vinci, but did lead to some headaches. Many paint and canvas salespeople kept knocking on da Vinci's door. Da Vinci started to think that Googlio was too focused on money when his search about the Last Supper resulted in caterers stopping him on the street and offering assistance.

Jeremy has an idea for Scott and Dave....I agree with the implied business context - that there is a business to be made in providing services like link spam blocking to serious bloggers. I'd pay a monthly fee for such a service.

As ClickZ points out, the Lycos 50 shows that searchers got serious last week, with war-related searches (Nick Berg in particular) beating the usual fluffernutter of Paris, Britney, and Clay by a huge margin - 12 times, Lycos reports.

Used to be, they did one thing, and one thing well. But, the world has changed. And Google has had to respond, yet continue to innovate, defend, as well as declare. Welcome to the second inning. Play Ball!

(too cryptic?! Pray for me to finish this fucking book.)

As I muddle my way through yet another iteration of my outline, and think about the issues raised in my recent ephemeral/eternal post, it seems apparent to me that as a culture we are nowhere near consensus on what rights, if any, a person has with regard to the data we create and/or provide to third party applications like A9, Gmail, Plaxo, and the like. Clearly we are touchy about all of this, as the reaction to Gmail proves. In the process of my research, I started reading the terms of service and privacy policies for various services, and found them inconsistent, often vague, and in general difficult to understand.

Now, I know there is a vocal contingent of folks who believe that we should simply assume we have no privacy online, and assume the quid pro quo for any service that we use is loss of control over the metadata/personal information we create along the way. I certainly understand this line of thinking, but...it strikes me as a cop out. In the end, I'd warrant that business models are going to evolve to the point where services will spring up that offers consumers access to their own clickstreams in new and powerful ways, and I'm going to predict that we will want that access as a right. I'd prefer we not have early lockdown on this issue, if we can at all avoid it.

The nice thing about doing a book is that people help you. I have had and continue to have help from a lot of smart folks, and one of them is Abigail Phillips, a lawyer who has worked with the CDT and the Berkman Center. Abigail is helping me pull together a little research project that will compare the policies of several well known platform players as they relate to what I'm calling "clickstream/stored information" - the data exhaust we all create when we interact with web-based services.

Now, I imagine this kind of work is ongoing at lots of places, and hopefully this lazyweb request will point me toward that work, if indeed it exists, as well as pertinent case law from the real world. In any case, we've tried to outline what the major issues are in the form of what we hope are clarifying questions. Below, I submit them to this readership for feedback and input. Once we get a good sample set - and we're trying to keep it simple, and avoid overly focused, complicated, specific or situational questions - we intend to review the Terms Of Service and Privacy Policies of four major services (we plan to start with an email provider, a major ecommerce player, a search site, and a social networking/contact site), and see what we learn.

If nothing else, we hope that we can report out a clearer sense of how each site "scores" on issues of consumer data protection and usage. That said, here are the questions, laid out in three rough categories of Ownership, Privacy/Usage, and Account Modification/Deletion. If you're into this kind of thing, please give them a read and post your responses. If not, stay tuned, and we'll report what we find out.

Thanks in advance!

-------

OWNERSHIP

Who owns the information-trail (clickstream) and/or stored personal information or profiles (stored information) created while using the service?

If the service owns it, does the user have any rights to view and/or edit that clickstream/stored information? Does the user have any rights to republish, aggregate, or profit from that information in other venues apart from the service where it was created?

Can the user transfer his or her clickstream/stored information to another web-based service? If so, can it be done easily, or is it a difficult and time-consuming task?

Does the service make it easy or difficult to access, edit, and/or retrieve copies of the user’s clickstream/stored information?

PRIVACY/USAGE

Who has access to the clickstream/stored information that a user posts or creates on the site?

Is there a place where the service outlines and regularly updates exactly how it uses this information? Is there a reasonable mechanism for the user to request and receive information on such use?

What is the strategic role of such information in the ongoing business/service, both specifically to the service and more generally to the larger business?

Does the site transfer to third parties personal data that the user submits to or creates on the site? If so, is it connected to specific user profiles, or is it delivered in aggregate form?

Under what circumstances (request, subpoena, etc) will the clickstream/stored information be released to law enforcement or government entities?

ACCOUNT MODIFICATION/DELETION

Does the service have the right to delete an account and all related information without notice to the affected user?

When a user deletes information from an account, is it deleted from the service’s servers and any backups the service may have? If not, does the user have recourse to insure permanent deletion?

If the user closes an account, does the service delete all copies of the information that is stored in the account? Do all third parties that have received user information through the service delete that information?

What happens to user information in the event the user dies while the account is still active? If the user owns that information, or has rights to that information, can those rights be transferred those rights to others, such as an estate or family?

What guarantees do users have that their information will be protected if the service is sold to another company?

What is the service’s policy as it relates to altering its terms of service/privacy policies? Will a user be notified prior to such changes, and will the users have a period of time to react prior to those changes taking effect?

OK, I know, I'm supposed to be writing. But when IBM takes a shot like this across Google's bow, I have to point it out. Ben Elgin, of Businessweek, heads down to WebFountain (yes, that WebFountain) and interviews the two chief geeks. During the course of the interview, Elgin asks if Google and WebFountain are similar. Dan Gruhl, WebFountain's chief architect, answers:

We crawl continuously. We look to be updating our store [of information] within 20 minutes of when a page [on some other Web site] changes. We'd be happy to answer two or three questions per minute that are very complex and change how business works. That's a very different target market. It's kind of like saying: Yeah, strictly speaking a Rolls Royce Phantom and Yugo are both cars. But the fact is, they're serving very different markets.

Oh my. I sense a certain frustration with all the attention those folks up in Mountain View have been getting lately.

Don't expect a hell of a lot of news during the quiet period.

(Thanks Gary)

Astute Searchblog reader Rohit Khare points me to this odd passage on page 77 of the Google S1:

2003 Equity Incentive Plan
Our 2003 Equity Incentive Plan was assumed by us in connection with our acquisition of Ignite Logic, Inc. in April 2004. At April 23, 2004, options to purchase a total of  (blank)  shares of Class A common stock were....

Well, the purchase was in fact covered by a local paper (the firm is based near Sacramento, Ca.).

What is Ignite Logic, Inc.? It's a startup which helps law firms set up web sites. But why buy them? Hey, if you have good tech and processes to get a law firm's site up and running, one might imagine it just might scale to the entire SMB market (and beyond). Recall my "Incubation Platform" post - about how Google might leverage its platform to let others build on top of their infrastructure? You thought I was smoking something, eh? Nope.

Congrats to the team there, which has shipped a major upgrade. Details on new features here. Main stuff: Comments are enabled, a new "profile" page for Bloggers (watch that space, will be important in the building of meta-Web stuff), stronger permalink support, conditional tagging, email blogging, etc.

Hey Folks - Late Monday I'll send out invites to the conference to all those who sign up to the newsletter at left, or you can still request an invite by heading here. That's when I'll send out the weekly Searchblog newsletter as well. Have a great weekend!

John Brockman republishes a four-year old essay from Danny Hillis positing "Aristotle," a tutor program built around a "knowledge web" (not unlike the semantic web, but more specialized) which might revolutionize how we learn. Many luminaries weigh in on the concept. Not light reading, but interesting, and very search-driven. Neal Stephenson fans will hear an echo of "The Primer" from The Diamond Age.

A fellow by the name of Stephen Morrison has hacked up "Proogle," a Google skin that returns Google's results but adds in PageRank scores. The site is linked to what I presume is Stephen's home site, Webmaster Brain" (no contact info on his site, but a number of neat tools are there, including a link popularity tester).

I'm told Proogle has gotten quite popular among the webmaster community, as a result, I'll wager won't be up for long - it more likely than not generates more than 1000 searches a day, a violation of Google's terms of service. (The site even implores: "Google, Please Don't Sue!") This is yet another example of interesting hacks built on top of Google that, in the end, will probably end up on the dustbin due to popularity. Another recent example is Social Grid. I did hear back from the fellow behind that site, who admits he has yet to "ask permission" to build on top of Google. His credo: Code now, ask for forgiveness later.

Thanks to Aaron Wall of SEO Book/ SEO Index for pointing this out to me.

Wired News reports on a study done at Brandeis. They coulda just asked Technorati or Feedster.

(Part 1 of …?)
This is an idea I'm starting to rough out. As I said earlier, I will be testing your patience over the coming weeks as I do this more frequently. These essays are not intended to be the “book,” but rather sketches that lead to the book. (Lord knows, I can’t assume a general readership will be nearly as forgiving as you hardy souls have proven to be.)

I'm interested in what I'll call the shift from the ephemeral to the eternal. Gmail is a good example of this, as are Plaxo, social networks, and most ecommerce sites that keep profiles of our browsing and buying habits. And search - in particular, the approach to search that A9 has taken - is perhaps the most interesting and difficult to classify expression of the trend.

In the past few years, a good portion of our digitally mediated behavior – be it in email, search, or the relationships we have with others – has become eternal – in other words, recorded and preserved by one entity or another, usually commercial in nature. And as this information has become eternal, we, as creators of that information, have lost a large degree of control over how that information is used and in what context. In fact, in many cases we have lost ownership of the information altogether – arguably before we even knew it existed in the first place. Whether this matters at all is worth debate – after all, how could we lose that which we never had? It’s not my goal to write a privacy screed here, nor take “evil corporations” to task. But it seems to me the issues raised by the ownership of our collective data exhaust are certainly worth raising and discussing, with a particular eye toward the Law of Unintended Consequences, if nothing else.

So cast yourself back to the pre web days, the “PC era” of 1985-1995. In this phase of the computing revolution, we brought our habitual presumptions to communication and discovery via the computer keyboard – we assumed (rightly or wrongly) that there was no permanent record of our actions on the computer. When we rummaged through our hard drives or, later, across LANs and WANs, we assumed the digital footprints we left behind – our clickstream exhaust, so to speak - were as ephemeral as a phone call - fleeting, passing, unrecorded. Why would it be anything but? Clickstreams had no value beyond the action they predicated, serving only as a means to an end of finding a file or passing along a message.

The same assumptions clothed our email. Sure, we understood that email might reside (briefly) on servers, but for years we assumed they were our emails, and the ISP or network over which they passed had no rights to examine or manipulate them, much less own them. (In fact, the Electronic Communications Privacy Act of 1986 codified this sentiment into law, at least for private email). While the more sophisticated email user among us has grown to understand that the folly of this assumption in a corporate environment, the idea that email is an ephemeral medium is still widely held. Just this week Frank Quattrone, one of the technology sectors’ most powerful players and hardly a rube, was brought down by such a presumption.

But for most of us, the possibility of such negative consequences is remote, and I’d argue that most users still believe email is an intensely private and ephemeral form of communication. And this holds true even when that email lives on the servers of yahoo.com, hotmail.com, or gmail.com.

As for social networks, in the PC era the very idea that our relationships to others (our social network) or our relationships to goods and services (our commercial network) were anything but ephemeral was presumed: without the internet, how could it be otherwise? Sure, once in a long while someone got a hold of your calling card or your credit card slip, and your privacy and security were breached, but like email, the chances of this occurring were so minute as to be irrelevant . And before Friendster, “social networks” were simply records in your private contact database.

In short, before the web, we could pretty safely assume that our hard drive rummaging, our email, and our networking habits – in short, our clickstream – were ephemeral, known only to us (and soon forgotten by us, I’d wager). 
 
But as I've posted before, and as many have noted before me, as an internet culture we are steadily moving our ephemeral habits from the desktop to the web, and from our local control to the servers of corporations. (I was not surprised to learn yesterday that one in ten internet users have registered at a social network, for example, and one in five have visited one). The reason for this shift is simple: innovative companies have figured out how to deliver great services (and make money) by divining clickstream patterns, be it a underlying divination, like PageRank, or a more direct one, such as AdWords or Amazon’s recommendation system. And from a consumers’ point of view, there are also very simple and compelling reasons for this shift: services like search, Plaxo and Gmail make our lives easier, faster and more convenient. But as we move our data from the edges to the center, a question arises: Have our assumptions moved with our data?   

Brad Templeton, among many others, offers perhaps the most reasonable assessment of this question as it relates to Gmail (his conclusion: we need to revise our assumptions about privacy and ownership), but I’d argue that the issues he raises can be more broadly discussed in relation to search.

Search provides a framework for thinking not only about mail (what is it about Gmail that makes it really unique? It’s searchable…), but for our entire clickstream, which is fast becoming an asset - certainly to the individual, but in particular to the internet industry. Search drives clickstreams, and clickstreams drive profits. To profit in the internet space, corporations need access to clickstreams. And this, more than any other reason, is why clickstreams are becoming eternal. As we root around in the global information space, search has become our spade, the point of our inquiry and discovery. The empty box and blinking cursor presage your next digital artifact, the virgin blue link over which your mouse hovers will transform into one more footprint through this era’s Olduvain ash.

But once eternal, what then? Beyond commerce, what happens to our culture when the previously unknown becomes knowable, and, to ping Kevin Kelly, out of control? I’d love to hear what you think, I’d guess that the consequences are pretty far reaching.

(Part 2, coming soon: “Whose Data? – Examples of Clickstream Creation (A9, Furl, et al) and the Tricky Issues They Raise.” At least, I think that’s what is coming next…)

From Comscore, via IP:

* SEARCH VOLUME: In total, Americans conduct between 3.0 and 3.5
billion searches per month. More than one billion of these searches are
typically conducted at Google. The average search engine user conducted 32
searches in February. The average Google user conducted 25 searches at the
engine, more than twice the average number of searches (12) conducted by
users of the top ten engines. (Source: comScore qSearch)

* SHARE OF SEARCH: In February, Google controlled approximately 35
percent of searches conducted at major search engines by U.S. Internet
users. Yahoo!, its closest competitor, conducted 30 percent of searches by
U.S. Internet users in February. Among worldwide Internet users (Anglophone
population), Google's lead is even more dramatic, with the company
accounting for more than 43 percent of all searches. (Source: comScore
qSearch)

Share of Online Searches
February 2004
Source: comScore qSearch

U.S. Internet Users / Worldwide Internet Users
(Anglophone population)
Google Sites 34.7% / 43.3%
Yahoo! Sites 30.0% / 30.8%
MSN-Microsoft Sites 15.4% / 14.1%
Time Warner Network 15.0% / 7.1%
Ask Jeeves 1.9% / 1.7%
All Other 3.0% / 3.0%

* PENETRATION: Approximately 50 percent of all U.S. search engine
users conducted at least one search at Yahoo! in February, the highest
penetration of any search engine. Google ranked second, conducting at least
one search for 45 percent of all U.S. search engine users. (Source: comScore
qSearch)

* UNIQUE VISITORS: Combined, search sites reach more than 130 million
Americans or approximately 85 percent of all Internet users each month,
ranking Search/Navigation among the most popular categories on the Web.
Just over 65 million people visited Google sites in March 2004, an increase
of 23.5 percent versus March 2003. (Source: comScore Media Metrix)

Unique Visitors and Audience Reach
March 2004
Source: comScore Media Metrix
U.S. Home, Work and University Internet Users

Unique Visitors / Reach %
(000)

Total Internet Users 154,051 / 100
Search/Navigation Category 131,030 / 85.1

Google Sites 65,029 / 42.2
Yahoo! Search 59,755 / 38.8
MSN Search 48,912 31.8
AOL Proprietary Search 34,629 / 22.5
Ask Jeeves 17,524 / 11.4
Lycos Network Search 9,024 / 5.9

* PROPERTY BREAKDOWN: Below is a breakdown of traffic to Google's
network of sites. With the March introduction of Froogle as one of the tabs
on the Google home page, traffic to this section of the site is likely to
increase quickly in the coming months. (Source: comScore Media Metrix)

Property Breakdown: Google Sites
March 2004
Source: comScore Media Metrix
U.S. Home, Work and University Internet Users

Unique Visitors
(000)

Google Sites 65,029
GOOGLE.COM 63,057
Google Web Search 58,494
Google Images 15,924
Google Directory 5,074
Google News 3,277
Google Groups 1,954
Froogle 749
BLOGSPOT.COM 3,382

If any of you are baseball fans...is there some way we can publicly protest this most short-sighted of developments? I mean, Spiderman ads on the fucking bases??? I know it's getting harder and harder for advertisers to get their message out to large audiences, but...Spiderman ads on the fucking bases??? Quite possibly the stupidest thing I have seen from the marketing cartel in quite some time.

UPDATE: Happy day, they pulled it after one day of public comment! I think that qualifies it as a major fuckup, rather than a smart marketing play.

A big thanks to CNBC Valley corresondent and Searchblog buddy Cory Johnson for this file (will download upon click), which takes all the financial information found in Google's S1 filing and pours it into a handy spreadsheet form.

Cory has broken out Applied Semantics, the one acquisition with related financial information. As he says:

Google paid $41.5 million and 1,825,226 fully-vested shares, as well as
557,574 "fully-vested and unvested options" of Google stock valued at $60.9
million (that'd be $109 an option).

Interesting idea. Social Grid. Does this violate the Google API? (I've sent the owner an email.) This is, however, a good example of how others can build on top of the search platform. From Internetnews.com:

The idea is simple. Registered members turn their own personal or business Web pages into free personals ads.

"It's a Google for people," said SocialGrid founder Chau Vuong.

Apparently, he's hoping to be acquired, as he has a rather self defeating approach to his own business:

With the search wars in overdrive, Vuong thinks that Google should continue to expand. "Sooner or later, Google is going to come out with their own HotJobs and GeoCities," he said. "Orkut is like Google's answer to Yahoo Groups. SocialGrid is basically a Google's version of Yahoo's GeoCities and HotJobs."

I was thinking about what Google might do with the huge platform it has and is continuing to build. What might be a profitable and deeply cool use of such a platform? Something Wayne Rosing said in Alex's piece struck me, when crossed with Simson's Akamai insights:

Engineering Vice-President Wayne Rosing has on several occasions emphasized that Google's primary expertise is in so-called distributed computing. That's a fancy way of talking about delivering applications to a computer user's browser or to remote locations.

So, what if Google becomes an application server cum platform for business innovation? I mean, a service, a platform service, that any business can build upon? In other words, an ecologic potentiality - "Hey guys, over here at Google Business Services Inc. we've got the entire web in RAM and the ability to mirror your data across the web to any location in real time. We've got plug in services like search, email, social networking, and commerce clearing, not to mention a shitload of bandwidth and storage, cheap. So...what do you want to build today?"

If I had that opportunity, I'd take a percentage of revs or profits on the businesses that got built, rather than just service fees. it's Google as incubator to Web 2.0.

Yahoo is already doing this, though for a fee and in the SMB market. So is MSN. The traces are laid. Both of them were also doing mail. But neither of them have more than 100,000 servers and the GFS. Hmmmm.

OK, back to writing the book, damnit.

The ideas herein complement some of those which I'm pondering in my "From the Ephemeral to the Eternal" riff, which I might post here soon, if I can stop getting distracted by reading stuff like this. The whole privacy debate is a miasma, and I'm struggling with having something new to say without having to go back and spend 5000 words on history and context. On that note, there's this new book in the mix to boot.

Alex reports on Google's 2004 capital spend and what it augurs.

Really bad idea.

Fellow Searchbloggers -

I'm hunkering down over the next month or two, and focusing on two things: my book, first and foremost, and the conference. Something has to be dialed back, and it will most likely be the stream of postings that hit this page every day. I will likely post more musings on the book, and less news and news analysis. Not to worry, though, once I make some progress on the book, and the initial program is set for the conference, I'll be back at it full force, most likely in a new and improved fashion.

For good coverage of news, I suggest you add Gary Price and Andy Beals to your must read list, along with Danny and Chris over at SEW, of course.

Keep checking here, though. I know I won't be able to stop posting on *really* important news, and I'll be soliciting your feedback on my longer musings that are book related on (hopefully) a very regular basis.

Mitch (founder of Lotus and the EFF, square that circle...) has seen some shit. Here's his take.

Excerpts:

This insulation from shareholder pressure has been criticized as arrogant and anti-democratic, but it is also prudent, I'm sad to say, since Google intends to run its business with an eye to risky, creative experiments that are poorly understood or tolerated by the public markets.

...At least the Google guys want to do the right thing in terms of responsible corporate citizenship and for this they should get a lot of credit. What needs to change are the basic terms of the deal between public companies and the public. Business success needs to be measured not just by profit, but by social impact as well. In a world where that was the norm, absolute returns on stocks might be slightly lower, but absolutely everyone, would be better off.

This is really worth a read .
Excerpts:

So there you had Google coming out rightfully declaring that any incremental value generated by investors belongs to Google, at the same time as the broker dealer mechanism was busy trying to steer basic equity trading business through its desk. I believe that Google's decision to set its offering price through a retail auction mechanism has the potential of destroying a traditional profit center of wall street. ...

...what I saw was the end of a certain kind of investment banking innocence. No, the outsized commissions are not your divine right. No, you can't control the allocation of underpriced shares to your best clients. Yes you will be paid, but it will be in fees like those paid to lawyers, consultants or accountants. Profiting off of outsized bid-ask spreads would have to be replaced by a different type of value. I am not sure Wall Street has figured out what it will do if Google's auction model proves to become the rule rather than the exception from here on in.

Seth runs a very interesting research company called Majestic Research.

Today we announced Web 2.0, a conference for and about the internet industry. I'm teaming with the folks at O'Reilly and MediaLive on this event. It's in October 5-7 in San Francisco, at the Hotel Nikko. The conference is limited to an invitation list, but it's loosely joined. If you're interested, let me know by signing up for the newsletter at left (I'll be sending a note about the conference to that list later in the month), or go to the site and request an invite and say you heard about it through Searchblog.

While it's still early on in the program, I'm pleased to say we've got a great lineup of folks speaking, including Jeff Bezos, Mark Cuban, John Doerr, Mitch Kapor, Larry Lessig, Mary Meeker, and Dan Rosensweig.

I met with Damian Smith, who came from Australia to run Looksmart on an interim basis this January, a month or so ago. I didn't post on it here, mainly because our conversation was book related or topics I imagined he didn't want posted all over the net. But he sent me a note Friday regarding the two Looksmart news items last week, and they do merit attention. First, the company bought Net Nanny. From the release:

Net Nanny is the world's leading brand name in consumer online filtering.

"This acquisition is both strategic and prudent for LookSmart," said Damian Smith, LookSmart's CEO. "Strategic, because integrating our search technology into Net Nanny provides a stronger product for their users, while also providing LookSmart with a desktop platform from which to launch high margin search and paid listings applications. Prudent, because Net Nanny is expected to produce positive margin contribution for LookSmart in 2004."

Second, the company announced better earnings than were expected - and raised guidance for next year.

KK points me to this post from Tristan Louis which reverse engineers how many machines Google has based on reported IT costs. Answer: roughly 50-80K machines, each with two CPUs. Honestly, I think it's more. In any case, Tristan makes the case that taken together, it's the most powerful computer in the world. Anyone care to disagree? I'd be interested to hear if IBM agrees.

Kartoo has launched new search engine, Ujiko. I am sure there is an explanation for this name, but I don't know it. Uses Yahoo Search technology, and lets you to customize results based on what you like and don't like, and, interestingly, keeps all user data on *your* machine. I do believe this issue of who controls/owns/has rights to user data will be important to most users in a few years, and a significant point of product differentiation. Gary has a review here.