free html hit counter Joints After Midnight & Rants Archives - Page 12 of 43 - John Battelle's Search Blog

Google Responds: No,That’s Not How Facebook Deal Went Down (Oh, And I Say: The Search Paradigm Is Broken)

By - January 13, 2012

(image) I’ve just been sent an official response from Google to the updated version of my story posted yesterday (Compete To Death, or Cooperate to Compete?). In that story, I reported about 2009 negotiations over incorporation of Facebook data into Google search. I quoted a source familiar with the negotiations on the Facebook side, who told me  “Senior executives at Google insisted that for technical reasons all information would need to be public and available to all,” and “The only reason Facebook has a Bing integration and not a Google integration is that Bing agreed to terms for protecting user privacy that Google would not.”

I’ve now had conversations with a source familiar with Google’s side of the story, and to say the company disagrees with how Facebook characterized the negotiations is to put it mildly. I’ve also spoken to my Facebook source, who has clarified some nuance as well. To get started, here’s the official, on the record statement, from Rachel Whetstone, SVP Global Communications and Public Affairs:

“We want to set the record straight. In 2009, we were negotiating with Facebook over access to its data, as has been reported.  To claim that the we couldn’t reach an agreement because Google wanted to make private data publicly available is simply untrue.”

My source familiar with Google’s side of the story goes further, and gave me more detail on why the deal went south, at least from Google’s point of view. According to this source, as part of the deal terms Facebook insisted that Google agree to not use publicly available Facebook information to build out a “social service.” The two sides had already agreed that Google would not use Facebook’s firehose (or private) data to build such a service, my source says.

So what does “publicly available” mean? Well, that’d be Facebook pages that any search engine can crawl – information on Facebook that people *want* search engines to know about. This is compared to the firehose data that was the core asset being discussed between the parties. This firehose data is what Google would need in order to surface personal Facebook pages relevant to you in the context of a search query. (So, for example, if you were my friend on Facebook, and you searched for “Battelle soccer” on Google, then with the proposed deal, you’d see pictures of my kids’ soccer games that I had posted to Facebook).

Apparently, Google believed that Facebook’s demand around public information could be interpreted  as applying to how Google’s own search service was delivered, not to mention how it (or other products) might evolve. Interpretation is always where the devil is in these deals. Who’s to say, after all, that Google’s “social search” is not a “social service”? And Google Pages, Maps, etc. – those are arguably social in nature, or will be in the future.

Google balked at this language, and the deal fell apart. My Google source also disputes the claim that Google balked at being able to technically separate public from private data. Conversely, my Facebook source counters that the real issue of public vs. private had to do with Google’s refusal to honor changes in privacy settings over time – for example, if I deleted those soccer pictures, they should also be deleted from Google’s index. There’s a point where this all devolves to she said/he said, because the deal never happened, and to be honest, there are larger points to make.

So let’s start with this: If Facebook indeed demanded that Google not use publicly available Facebook data, it’s certainly understandable why Google wouldn’t agree to the deal. It may not seem obvious, but there is an awful lot of publicly available Facebook pages and data out there. Starbucks, for example, is more than happy to let anyone see its Facebook page, no matter if you’re logged in or not. And then there’s all that Facebook open graph data out on the public web – tons of sites show Facebook status updates, like counts and so on in a public fashion. In short, asking Google to not leverage that data in anything that might constitute a “social service” is anathema to a company who claims its mission to crawl all publicly available information, organize it, and make it available.

It’s one thing to ask that Google not use Facebook’s own social graph and private data to build new social services – after all, the social graph is Facebook’s crown jewels. But it’s quite another thing to ask Google to ignore other public information completely.

From Google’s point of view, Facebook was crippling future products and services that Google might create, which was tantamount to an insurance policy of sorts that Google wouldn’t become a strong competitor, at least not one that  leverages public information from Facebook. Google balked. If Facebook’s demand could have been interpreted as also applying to Google’s search results, well, that’s a stone cold deal killer.

I certainly understand why Facebook might ask for what they did, it’s not crazy. Google might well have responded by narrowing the deal, saying “Fine, you don’t build a search engine, and we won’t build a social network. But we should have the right to create other kinds of social services.” As far as I know, Google didn’t chose to say that. (Microsoft apparently did). And I think I know why: The two companies realized they were dancing on the head of a pin. Search = social, social = search. They couldn’t figure out a way to tease the two apart. Microsoft has cast its lot with Facebook, Google, not so much.

When high stakes deals fall apart, both sides usually claim the other is at fault, and that certainly seems to be the case here. It’s also the case with the Twitter deal, which I’ve gotten a fair amount of new information about as well. I hope to dig into that in another post. For now, I want to pull back a second and comment on what I think is really going on here, at least from the perspective of a longer view.

Our Cherished Search Paradigm Is Broken (But We Will Fix It….Eventually)

I think what we have here is a clear indication that the search paradigm we’ve operated under for a decade or so is broken. That paradigm stems from Google’s original letter to shareholders in 2004. Remember this line?Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating.

In many cases, it’s simply naive to claim Google is unbiased or objective. Google often favors its own properties over others, as Danny points out in Real-Life Examples Of How Google’s “Search Plus” Pushes Google+ Over Relevancy and others have also detailed. But there is a reason: if you’re going to show results from all other possible contenders, replete with their associated UI and functional bells and whistles (as Google does with its own Maps, Pages, Plus etc.), well, it’s nearly impossible now to determine which service is the right answer to a particular person’s query. Not to mention, you need to put a deal in place to get all the functionality of the service. Instead, Google has opted, in many cases, to go with their own stuff.

This is not a new idea, by the way. Yahoo’s been doing it this way from the beginning. The contentious issue is that biasing some results toward Google’s own products runs counter to Google’s founding philosophy.

I have a theory as to why all this is happening, and I don’t entirely blame Google. Back when search wasn’t personalized, Google could defensibly say that one service was better than another because it got more traffic, was linked to more (better PageRank), and so on. Back when everyone got the same results and the web was one homogenous glob of HTML, well, you could claim “this is the best result for the general population.” But personalized search has broken that framework – I lamented this back in 2008 with this post: Search Was Our Social Glue. But That Is Dissolving (more here).

With the rise of Facebook and the app economy, the problem of search has become terribly complicated. If you want to have results from Facebook in your search, well, that search service has to do a deal with Facebook. But what if you want results from your running app (I have hundreds of rides and runs logged on AllSportGPS, for example)? Or Instagram? Or Path, for that matter? Do they all have to do deals with Google and Bing? There are so many unconnected pieces of the Internet now (millions of apps, most of our own Facebook experiences, etc. etc.) that what’s a good personal result for one person is not necessarily good for another. If Google is to stay true to its original mission, it needs a new framework and a massive number of new signals – new glue – to put the pieces back together.

There are several ways to resolve this, and in another post, I hope to explore them (one of them, of course, is simply that everyone should just go through Facebook. That’s the vision of Open Graph). But for now, I’m just going to say this: The issues raised by this kerfuffle are far larger than Google vs. Facebook, or Google vs. Twitter. We are in the midst of a major search paradigm shift, and there will be far more tears before it gets resolved. But resolve it must, and resolve it will.

  • Content Marquee

Compete To Death, or Cooperate to Compete?

By - January 11, 2012

(image) **Updated at 3 PM PST with more info about Facebook/Google negotiations…please read to the bottom…**

In today’s business climate, it’s not normal for corporations to cooperate with each other when it comes to sharing core assets. In fact, it’s rather unusual. Even when businesses do share, it’s usually for some ulterior motive, a laying of groundwork for future chess moves which insure eventual domination over the competition.

Such is the way of business, particularly at the highest and largest levels, such as those now inhabited by top Internet players.

Allow me to posit that this philosophy is going to change over the next few decades, and further, indulge me as I try to apply a new approach to a very present case study: That of Google, Facebook, and Twitter as it relates to Google’s search index and the two social services’ valuable social interaction datasets.

This may take a while, and I will most likely get a fair bit wrong. But it seems worth a shot, so if you feel like settling in for some Thinking Out Loud, please come along.

First, some abridged background. Back in 2009, on the Web 2 Summit stage of all places (yes, I was the emcee), Google, Microsoft, Facebook and Twitter announced a flurry of deals, some of which were worked out in a last minute fury of negotiations. Early in the conference Microsoft announced it would incorporate Twitter and Facebook feeds into its new search engine Bing. Not to be outdone, Google announced a deal with Twitter the next day. However, Google did not announce a deal with Facebook, and the two companies have never come to terms. Meanwhile, Microsoft has continued to deepen its relationship with Facebook data, to the point of viewing that relationship as a key differentiator between Bing and Google search.

All of these deals have business terms, some of them financial, all with limits on how data is used and presented, I would presume. Marissa Mayer of Google told me on the Web 2 stage that there were “financial terms” in Google’s deal with Twitter, but would not give me any details (nor should she have, frankly).

Fast forward to the middle of last year, when the Google/Twitter deal was set to expire. At about the same time as renewal was being negotiated, Google launched Google+, a clear Facebook and Twitter competitor. For reasons that seem in dispute (Google said yesterday Twitter walked away, Twitter has not made a public statement about why things fell apart), the renewal never happened.

And then yesterday, Google incorporated Google+  results into its main search index, sparking a debate in the blogosphere that rages on today – Is Google acting like a monopolist? Does Facebook or Twitter have a “right” to be included in Google results? Why didn’t Google try to negotiate inclusion with its rivals prior to making such a clearly self-serving move?

Google execs, including Chair Eric Schmidt, told SEL’s Danny Sullivan that the company would be happy to talk to both companies to figure out ways to incorporate Twitter and Facebook into Google search, but clearly, those talks could have happened prior to the G+ launch, and they didn’t (or they did, and did not work out – I honestly have no idea). When Danny pointed out that Twitter pages are publicly available, Schmidt demurred, saying that Google prefers to “have a conversation” with a company before using its pages in such a wholesale fashion (er, so did they have one, or not? Anyway…). He has a point (commercial deals are de-rigueur), but…that conversation happened last year, and apparently ended without a deal. And around we go…

What’s clear is this: All the companies involved in this great data spat are acting in what they believe to be their own self interest, and the greatest potential loser, at least in the short term, is the search consumer, who will not be seeing “all the world’s information” but rather “that information which is readily available to Google on terms Google prefers.”

The key to that last sentence is the phrase “what they believe to be their own self interest.” Because I think there’s an argument that, in fact, their true self interest is to open up and share with each other.

Am I nuts? Perhaps. But indulge my insanity for a bit.

The Cost of Blinkered Competition

Back in the Web 1.0 days, when I was running The Industry Standard, I had a number of strong competitors. It’s probably fair to say we didn’t like each other much – we competed daily for news stories, advertiser dollars, and the loyalty of readers. The market for information about the tech industry was limited – there were only so many people interested in our products, and only so much time in the day for them to engage with us.

My strategy to win was clear: We’d make the best product, have the best people, and we’d win on quality. When I heard about one of our competitors badmouthing us, I’d try to ignore it – we were winning anyway: We had the dominant marketshare, the most revenues ($120mm in 2000, with $21mm in EBIDTA), and the best product.

Then something strange happened: an emissary from a competitor called and asked for a meeting. Intrigued, I took it, and was surprised by his offer: Let’s put our two companies together. Apart, he argued, we were simply tearing each other down. Together, we could consolidate the market and insure a long term win.

I considered his idea, but for various reasons, we didn’t take him up on it. I felt like we had the dominant position, that his offer was driven by weakness, not intellectual soundness, and I also felt that a combination would require that my shareholders take on too much dilution.

Two years later, both of us were out of business.

Now, I’m not sure it would have mattered, given the great crash of 2001. But what is certainly true is that I could have thought a bit deeper about what this fellow was proposing. Back in the days of print-bound information, we were essentially competing on what were publicly available assets: stories, particularly interpretations and reportage around those stories, and people: writers, editors, ad sales executives, and management. Short of combining companies, there wasn’t really any other way for us to collaborate, or at least, so I thought.

But perhaps there could have been. It’s been more than a decade since that meeting, and I still wonder: perhaps we could have shared back-end resources like operations, publishing contracts, etc. and saved tens of millions of dollars. We’d compete just on how we leveraged those public assets (stories, people). Perhaps we might have survived the wipeout of the dot com crash. We’ll never know. Since those publications died, the blogosphere has claimed the market, and now it’s far larger than the one we lost back in 2001. Of course I started Federated Media to participate in that model, and now FM has as large a revenue run rate as the Industry Standard, across a far more diverse market.

Why am I bringing this up? Because I think there’s a win-win in this whole Google/Facebook/Twitter dust up, but it’s going to take some Thinking Differently to make it happen.

Imagine Twitter and Facebook offer efficient access to all of their “public” pages – those that its users are happy to share with anyone (or even just to their pre-defined “circles”) – to Google under some set of reasonable usage terms. Financial terms would be minimal – perhaps just enough to cover the costs of serving such a large firehose of data to the search giant. Imagine further that Google, in return, agrees to incorporate this user data in a fashion that is fair – ie doesn’t favor any service over any other – be it Twitter, Google+, or Facebook.

Now, negotiating what is “fair” will be complicated, and honestly, should be subject to iteration as all parties learn usage patterns. And of course all this should be subject to consumer control – if I want to see only Twitter or Facebook or Google+ results in particular searches (or all results for that matter), I should have that right.

And this leads me to my point. Such a set up, regardless of how painful it might be to get right, would create a shared class of assets that would have to compete at the level of the consumer. In other words, the best service for the query wins.

That’s always been Google’s stated philosophy: the best answer for the question at hand. Danny gets to this point in a piece posted last night (which I just saw as I was writing this): Search Engines Should Be Like Santa From “Miracle On 34th Street”. In it he argues that Google’s great strength has been its pattern of sending people to its competitors. And he upbraids Google for violating that principle with its Google+ integration.

It doesn’t have to be this way. It’s not only Google that’s at fault here. Facebook won’t share with Google on any terms, Facebook and Google have not been able to come to terms on how to share data (more on that below*), and Twitter clearly wants some kind of value if it is to share its complete firehose with the search giant. Imagine if all three were to agree on minimal terms, creating a public commons of social data. Yes, that would put Google in an extreme position of trust (not to mention imperil its toddler Google+ service), but covenants can be put in place that allow parties to terminate sharing for clear breaches which demonstrate one party favoring itself over others.

Were such a public commons to be created, then the real competition could start: at the level of how each service interprets that data, and adds value to it in various ways.

Four years ago to the month, I wrote this post: It’s Time For Services on The Web to Compete On More Than Data

In it I said: It’s time that services on the web compete on more than just the data they aggregate….

I think in the end, Facebook will win based on the services it provides for that data. Set the data free, and it will come back to roost wherever it’s best used. And if Facebook doesn’t win that race, well, it’ll lose over time anyway. Such a move is entirely in line with the company’s nascent philosophy, and would be a massively popular move within the ouroborosphere (my name for all things Techmeme).

Compete on service, Facebook, it’s where the world is headed anyway!

Two and a half years ago, as it became clear Facebook’s “nascent philosophy” had changed (and as Twitter rose in stature), I followed up with this post: Google v. Facebook? What We Learn from Twitter. In that post, I said:

 

I think it’s a major strategic mistake to not offer (Facebook’s pages and social graph) to Google (and anyone else that wants to crawl it.) In fact, I’d argue that the right thing to do is to make just about everything possible available to Google to crawl, then sit back and watch while Google struggles with whether or not to “organize it and make it universally available.” A regular damned if you do, damned if you don’t scenario, that….

For an example of what I mean, look no further than Twitter. That service makes every single tweet available as a crawlable resource. And Google certainly is crawling Twitter pages, but the key thing to watch is whether the service is surfacing “superfresh” results when the query merits it. So far, the answer is a definitive NO.

Why?

Well, perhaps I’m being cynical, but I think it’s because Google doesn’t want to push massive value and traffic to Twitter without a business deal in place where it gets to monetize those real time results.

Is that “organizing the world’s information and making it universally available?” Well, no. At least, not yet.

By making all its information available to Google’s crawlers (and fixing its terrible URL structure in the process), Facebook could shine an awfully bright light on this interesting conflict (of) interest.

Thanks to Google’s inclusion of Google+ in its search index, that light has now been shone, and what we’re seeing isn’t all good. I’m of the opinion that a few years from now, each and every one of us will have the expectation and the right to incorporate our own social data into web-wide queries. If the key parties involved in search and social today don’t figure out a way to make that happen, well, they may end up just like The Industry Standard did back in 2001.
But not to worry, someone else will come along, pick up the pieces, and figure out how to play a more cooperative and federated game.
*Update: I’ve heard from a source with knowledge of the Facebook/Google negotiations over integration of Facebook’s data into Google’s search index. This source – who while very credible does come from Facebook’s side of the debate – explained to me that during the 2009 negotiations, Google balked at Facebook’s request that Facebook data be protected in the same fashion as it is in Facebook’s deal with Bing. In essence, Google claimed no way to keep data within circles of friends in the context of a Google search. According to this source: “Senior executives at Google insisted that for technical reasons all information would need to be public and available to all.” But the source goes on to point out that in Google’s own integration of Google+, Google does exactly what it claims it could not do with Facebook data. “The only reason Facebook has a Bing integration and not a Google integration is that Bing agreed to terms for protecting user privacy that Google would not,” this source told me.
Also, and quite interestingly, Google also refused to agree to a clause which stated that Google could not use the data to build its own social network. Now, this is where things can get very dicey. It’s very hard to prove whether or not a company is using the data in particular ways, and had Google agreed to that clause, it might have severely limited its ability to build Google+. What is clear is that Microsoft agreed to Facebook’s terms.

Search, Plus Your World, As Long As It’s Our World

By - January 10, 2012

Perusing my feeds today, I saw this post from Google’s blog:

Search, plus Your World

In the post, Google extols the virtues of incorporating results such as “your personal content or things shared with you by people you care about. These wonderful people and this rich personal content is currently missing from your search experience. Search is still limited to a universe of webpages created publicly, mostly by people you’ve never met. Today, we’re changing that by bringing your world, rich with people and information, into search.”

OH MY GOD! thinks I. GOOGLE IS FINALLY WORKING WITH FACEBOOK!

Nah, just kidding. What’s really going on is that Google is fully incorporating Google+ into its index. It’s as if Facebook doesn’t exist.

Now, I’ve been on this one before, and I’m sure others will point it out, or simply roll their eyes and call it a dead issue. Dead because we all know that Google hasn’t made peace with Facebook, and therefore is not crawling Facebook data, nor integrating Facebook results into its core search product in any other way than what’s absolutely necessary (ie those lame public Facebook profile pages). Facebook, in turn, has not made most of what happens inside Facebook available to search engines. It’s a standoff, because neither company really knows how to value the other company’s partnership.

And it sucks for the web. The unwillingness of Facebook and Google to share a public commons when it comes to the intersection of search and social is corrosive to the connective tissue of our shared culture. But as with all things Internet, we’ll just identify the damage and route around it. It’s just too bad we have to do that, and in the long run, it’s bad for Facebook, bad for Google, and bad for all of us. (BTW, Google also doesn’t show Twitter or Flickr results either, or any other “social” service. Just its own, Google+ and Picasa.)

Google addresses this issue in a SEL piece today:  “Facebook and Twitter and other services, basically, their terms of service don’t allow us to crawl them deeply and store things. Google+ is the only [network] that provides such a persistent service,” (said Google exec Amit) Singhal. “Of course, going forward, if others were willing to change, we’d look at designing things to see how it would work.”

Er, something tells me hell will freeze over first. Google’s already failed to get a data deal done with both Twitter and Facebook. I doubt they’ll take another run at it soon, though I wish they would.

Instead, we have the deepening trend of each of the Internet Big Five trying to be All Things to All People, creating a World That If Only You’d Use Exclusively, You’d Never Have To Leave.

Ick. Remember when Google used to be a neutral player that crawled the Whole Dern Web? So sad to see that era pass. It’s not Google’s fault, entirely, but it’s sad nonetheless.

NB: I should add that I am fully aware that the integration of G+, and *only* G+, into Google’s search service is a major win for Google’s fledgling social service. I’d expect a big bump in usage due to this, if the integration is done well (ie, doesn’t irritate users). It’s clearly “tying” in the sense of what Microsoft got slapped for in its DOJ antitrust case in the late 90s, but the context is different – Google doesn’t have a clear monopoly in search, just a pretty darn big one. If Microsoft really wanted to mess with Google, it could shut down Bing. Then Google might have some problems on its hands. Stranger things….

2011 Predictions: How Did I Do?

By - December 19, 2011

(image) For many years now I’ve made predictions, and for just as many years I review how I did. This is the week I do the reviewing, my predictions for 2012 should arrive around the New Year, assuming I find the right inspiration.

2011 was a strange year in many ways. We lost Steve Jobs, stupid Internet legislation reared its ugly head yet again in the form of SOPA, Internet IPOs came back in a big way (but didn’t perform as well as most would have liked), and the world woke up to the implications of programmatic buying and Big Data, in a Very Big Way.

As I look back on my predictions of twelve months ago, I think I did a pretty good job, but left plenty of room for improvement. Here’s a rundown of how I did, with some supporting citations, where appropriate:

Prediction #1: We’ll see the rise of a meme which I’ll call “The Web Reborn.” 

If you read this site closely, you’ll have noticed that I’m a bit up in arms about the impact this whole “AppWorld” phenomenon is having on the “real web.” AppWorld is not the Web, in fact, it’s utterly un-weblike. I’ve written about this all year long, too mostly little effect in the larger world. But just recently a meme has risen, in fact, that sounds an awful lot like what I predicted. To wit:

Dave Winer: Why apps are not the future and Enough with the apps already

Doc Searls: Broadband vs. Internet

Jonathan Zittrain: The PC is dead. Why no angry nerds?

Scott Hanselman: Apps are too much like 1990’s CD-ROMs and not enough like the Web

And of course, my latest: On This Whole “Web Is Dead” Meme

Score: I think I called this right. There is a robust movement toward saving the core principles that made the web what it is. But it’s at the early stages at this point. Score: 7 of 10. 

Prediction #2: Voice will become a critical interface for computing (especially mobile apps). 

Hello, Siri…I’d say this one happened, with bells on. And it’s just the beginning. Score this a 9 out of 10 (because Siri, while very important, is still not as good as it can be. Then again, Google, Amazon, and Microsoft all made voice moves this year as well).

Prediction 3: DSPs (Demand Side Platforms) will fade into the fabric of larger marketing platforms. 

This one is hard to quantify, it’s more of a feeling – DSPs were the Big Scary Development in the ad industry a year ago, and now…not so much. They are just part of the world we live in, and programmatic buying is a fact of life, one that is growing very, very quickly. Publishers have responded with their own adtech (FM purchased Lijit, for example, Google bought AdMeld), and on we go. I think I got this right, in the main.

Score: 8 of 10.

Prediction 4: MediaBank will emerge as a major independent player in the marketing world, playing off its cross channel reach (outside of digital) and providing an alternative to the conflicted digital platforms at Facebook, Microsoft, Google, and Yahoo.

Well, MediaBank merged with its main competitor, DDS, to form MediaOcean, run by the CEO of MediaBank. And the combined company is a behemoth, one capable of standing toe to toe with Google on several fronts. I think this one absolutely happened.

Score: 10 of 10.

Prediction 5: The Mac App Store will be a big hit, at least among Mac users, and may well propel Mac sales beyond expectations.

I’m not quite sure how to score this one. I don’t get the sense the Mac App Store was a big buzzy hit, but Apple recently released numbers about its success – 100 million downloads in the first six months of its existence (the last six months of this year).

And the company’s Mac sales are for sure on a roll, though I don’t know if one can attribute this to the Mac App Store. In short, it might be too early to call this one definitively, but net net, the trending is good.

Score: 7 out of 10.

Prediction 6: Apple will attempt to get better at social networking, fail, and cut a deal with Facebook.

Oh boy, I got this so tantalizingly right, or wrong…depending on how you score it. Let’s break it down. “Apple will attempt to get better at social networking…” Check – Apple introduced Ping late in 2010 and then updated it throughout the year.  “…fail…” Check. Ping never really took off. “…and cut a deal with Facebook.” DOOOOH. No way. If only I had said “…and cut a deal with Twitter….” I should have seen that Apple would never play nice with Facebook, if only due to the conflict around owning the business of media consumption. It sure did try, but negotiations broke down, according to most reports.

Score: 6 of 10.

Prediction 7: Apple will begin to show signs of the same problems that plagued Microsoft in the mid 90s, and Google in the past few years: Getting too big, too full of themselves, and too focused on their own prior success.

Now showing this prediction came true won’t be easy, but let’s take a few stories throughout the year as proof points. First, most of the “Web reborn” backlash (#1 above) is a response to Apple’s restrictive iOS. Then there’s the FT’s unqualified success in routing around Apple (see PC’s coverage), which nearly every major media and entertainment company is watching very closely. And there are plenty of rumblings (though not nearly as many as with Facebook, see #13):

IDC offers scathing prediction of certain death for Apple’s iAd program

Apple Made A Deal With The Devil (No, Worse: A Patent Troll)

Apple arrogance exposed by iPad undesign

All in all, I probably didn’t nail this one. But I’m not going to say it isn’t starting to happen. Score: 5 of 10.

Prediction 8: Microsoft will have a major change in leadership. I am not predicting Ballmer will leave, but I think he and the company will most likely bring in very senior new talent to open new markets or shift direction in important current markets.

Well, thanks to the Andy Lees move last week, I look like a genius. Also, long time Microsoft exec Yusuf Medhi, responsible for the most important piece of Microsoft’s new business (Bing), has been moved to its next possibly most important line of business, Xbox. Not to mention Ballmer’s various shakeups of senior management this year. Plus, rumblings continue about whether Ballmer is the right man for the job – to the point of some wags speculating that Gates might come back. As if. Score: 9 of 10. 

Prediction 9: The public markets will be surprisingly open to major new Internet deals.

Well, I could argue this is utterly true. After all, compared to 2010, there was a raft of Internet deals that got out this year – Demand, LinkedIn, Zillow, Angie’s List, Pandora, Zynga – with many more filings pending public debut. But none of them did very well – most if not all are trading below offering price. So the market let them get out, but isn’t trading them up. So was the market “open”? Yes. Was it enthusiastic after open? No.

Score: 7 of 10.

Prediction 10: The tablet market will have a year of incoherence. Apple will dominate with the iPad due to a lack of an alternative touchstone.

Well, that was way too easy. Exactly what happened – HP tried, failed. RIM tried, failed. Amazon is trying, and the early reviews ain’t great. We’ll see. Score: 10 of 10. 

Prediction 11: “Social deals” will morph to become a standard marketing outlet for all business, and by year’s end be seen as a standard part of any marketer’s media mix.

I think this has happened – nearly every small business has now heard of daily deals, and most forward leaning companies both large and small are using deals as a channel like any other (search, Yellow Pages, etc.). Don’t forget that a year ago, when I wrote this prediction, folks were talking about the deal market as something totally unique, like DSPs.

Related, in this prediction I also wrote “I’m tempted to say Facebook will abandon its own Deals offering for a deal with Groupon, but I’m not sure that will unfold in one year.” Facebook did indeed bail on its Deals offering, but did not cut a deal with Groupon. (Foursquare did).

Score: 9 of 10. 

Prediction 12: Groupon will fend off an acquisition by a major carrier, probably AT&T or Verizon. It’s possible they’ll sell, but I doubt it.

Google tried to buy Groupon around the time I made this prediction, but failed. I am unaware of any other suitors for the company on its way to its IPO, but I’m not privy to what might have happened. Nothing was publicly reported, however, so I’ll have to say I whiffed this one.

Score: 0 of 10.

Prediction 13: Facebook will decline as a force in the Internet world, as measured by buzz. The company will continue to be seen as Big Brother in the press, and struggle with internal issues related to growth. Also, it will lose some attention/share to upstarts. However, its share of marketing dollars and reach will increase.

OK, I think I got this one right, but one can argue, as always. Here are a litany of headlines over the year:

Amid backlash, Facebook tries to save face (Cnet)

Facebook as the New AOL (RWW)

Facebook Is AOLifying the Internet—and That Sucks (Gizmodo)

Facebook is gaslighting the web. We can fix it. (Anil Dash)

How Facebook is Killing Your Authenticity (Steve Cheney)

It’s time for a Facebook intervention (TNW)

President Obama Doesn’t Let His Daughters Use Facebook (HP)

The Decline and Fall of Facebook (Cringely)

I could find more, but I think you get a picture here of a broad swath of “buzz makers” questioning the company. Meanwhile, according to all reports, the company is killing it in revenue.

Score: 8 of 10. 

Prediction 14: We’ll see major privacy related legislation in the US brought to the floor of Congress, and then fail for lack of consensus. But that will drive a significant shift in how our culture understands its relationship to the world our industry is building, and that’s a good thing.

Well, we had quite a number of these bills brought to Congress in one way or another. I probably should not have used the words “to the floor,” as that implies open debate. Nearly everyone in Congress and the Executive branch talks about some kind of privacy legislation, but no one can seem to agree what that means.

As of late Fall, seven different legislators have introduced bills. But as far as I can tell, none have really gotten anywhere. However, the conversation around the issues has accelerated significantly, as predicted.

Score: 9 of 10. 

In summary, I think my predictions fared pretty well. If you take a score of 7 or more as a “hit,” 4-6 as a “foul ball,” and below 4 as a “strikeout,” I hit 10 of 14, fouled off 3 of 14, and whiffed massively just once.

So, how do you think I did?!

Related:

Predictions 2011

Predictions 2010

2010: How I Did

2009 Predictions

2009 How I Did

2008 Predictions

2008 How I Did

2007 Predictions

2007 How I Did

2006 Predictions

2006 How I Did

2005 Predictions

2005 How I Did

2004 Predictions

2004 How I Did

On This Whole “Web Is Dead” Meme

By - December 12, 2011

The Web is dead again, at least, according to a widely covered speech by Forrester Research’s George Colony.

Speaking at Le Web last week, Colony claimed that the HTML web is a poorly architected half step in the next, obvious progression of platforms: a hybrid between what we’ve come to know as the Web and the crippled chicletized place I’ve been calling AppWorld. In a stroke of nomenclature insight, Colony calls this new platform the App-Internet.

Colony argues that, unlike apps, the Web doesn’t leverage the processing and storage power of edge devices (like the iPad or a smartphone, for example. Or, say, an old school computer, like the one I’m using right now to write this post.)

In theory, I agree with Colony, but I’m not sure the real issues are elucidated in his speech, or in much of the coverage of it. If the next version of the “Web” loses that which makes the web special, it won’t be the web anymore. That’s when, for me anyway, the “web will be dead.” If the web keeps its core principles, well, then, we’ll keep calling it the web, I’d wager, even if it ends up looking like the “app-internet.”

Let me explain, if I can. I feel like I’ve written this many times before, but perhaps not so directly as now. What makes the web special is not the language(s) in which it is written  (HTML, javascript, and so on). To me, what makes the web special are the underlying assumptions about how it works. In no particular order, they include:

– No gatekeepers. The web is decentralized. Anyone can start a web site. No one has the authority (in a democracy, anyway) to stop you from putting up a shingle. Of course, we live in a society of law, so if that new site breaks a law, well, you might have to take that shingle down. That’s OK with me.

– An ethos of the commons. The web developed over time under an ethos of community development, and most of its core software and protocols are royalty free or open source (or both). There wasn’t early lockdown on what was and wasn’t allowed. This created chaos, shady operators, and plenty of dirt and dark alleys. But it also allowed extraordinary value to blossom in that roiling ecosystem. Over time, the good actors have come to agree on best practices which have allowed an extraordinary new resource to thrive. These practices include privacy policies, data use policies, and even some reasonable law (we’re still working out a lot of the law part, but it’s happening slowly and with consideration, as I believe it should). In short, we’re taking the time to codify the rules of the road on the web. We’ve not pushed it into early lockdown in terms of policy. The same is absolutely not true of AppWorld.

– No preset rules about how data is used. If one site collects information from or about a user of its site, that site has the right to do other things with that data, assuming, again, that it’s doing things that benefit all parties concerned. If it does stupid things, that site will be called out, and folks won’t visit it anymore. Yes, some people may get burned, but on balance, we’ve decided that it’s better to allow sites and their customers to determine the best use of data, rather than pre-determine data policy to the point where innovation is stifled. This is something of a corollary to my point above, but it bears explanation. Here’s one small example: When I watch a movie on Netflix using its website, Netflix stores information about how I watch that movie. If I stop watching and leave the site, Netflix remembers where I stopped, and gives me the option to resume the next time I go to the site. And if I fire up my iPad and launch the Netflix app, it also remembers where I was – because you can import data from the web into Apple’s AppWorld. But if I start watching a movie in AppWorld, then go to the open web to continue watching it, I can’t resume the movie. Netflix doesn’t remember what I’ve done on its iOS app. Why? Because in AppWorld, data can come in, but it can’t go out. This stifles innovation in so many ways I’m going to have to write another post to explain it.

– Neutrality. No one site on the web is any more or less accessible than any other site. If it’s on the web, you can find it and visit it. This is a corollary of “no gatekeepers,” but again, it bears elucidation. In current versions of AppWorld, finding anything is a challenge, and the winners are almost always those who get special treatment in a gatekeeper’s storefront.

– Interoperability. Sites on the web share common protocols and principles, and determine independently how to work with each other. There is no centralized authority which decides who can work with who, in what way. In AppWorld, the apps don’t talk to each other, and from what I can tell, you need permission from the gatekeeper if you’re going to work around that fact. Again, this stifles true innovation. Imagine if Google needed permission from some overlord to crawl the web back in 1998. It would never have gotten off the ground, and the second great Web boom may never have happened.

I’m sure I’ve missed a few key points, but I think you get the picture. If Colony’s “app-internet” sheds the crappy parts of AppWorld, and keeps the best parts of our current Web, I’m all for it. But if the reverse happens, I’m all in – against it. Long live the web!

Neal Stephenson on Important Work

By -

(image) An interesting interview in the NYT  I missed from last week, with noted author Neal Stephenson. In it, he riffs on something that’s been bugging me as I work on the book. Asked about “the future of computing,” he responds:

“I’ll tell you what I’d like to see happen,” he said, and began discussing what the future was supposed to have looked like, back in his 1960s childhood. He ticked off the tropes of what he called “techno-optimistic science fiction,” including flying cars and jetpacks. And then computers went from being things that filled a room to things that could fit on a desk, and the economy and industries changed. “The kinds of super-bright, hardworking geeky people who, 50 years ago, would have been building moon rockets or hydrogen bombs or what have you have ended up working in the computer industry, doing jobs that in many cases seem kind of ignominious by comparison.”

Again, a beat. A consideration, perhaps, that he is talking about the core readership for his best sellers. No matter. He’s rolling. He presses on.

“What I’m kind of hoping is that this is just kind of a pause, while we assimilate this gigantic new thing, ubiquitous computing and the Internet. And that at some point we’ll turn around and say, ‘Well, that was interesting — we have a whole set of new tools and capabilities that we didn’t have before the whole computer/Internet thing came along.’ ”

He said people should say, “Now let’s get back to work doing interesting and useful things.”

I know that many of us in the Internet industry believe we are working on things that are changing the world. But it’s worth asking ourselves if honestly that’s true. We’ve got some really big problems to attack, and we need the best minds of our generation on them.

Stephenson’s thoughts are elucidated in more detail in a piece he wrote for World Policy here.

Instrumenting People Into Location Services

By - December 08, 2011

So this week a well known VC made the trek to my writing retreat in Marin, and we hung out in a room that until this year was a large storage closet behind my garage. I rethought the space, soundproofed it, added a hodge-podge of AV gear and musical instruments, and named the place the “Ross Social Club”  – on Foursquare, anyway. I haven’t really told anyone that I gave the place a name, but it was sort of an experiment – would anyone ever check in there besides me?

Now I chose that name for various reasons I won’t get into here (another story, one I’ll be glad to tell you over a bourbon). But I like being able to name a space on Foursquare, and it’s become a habit for me to “check in” whenever I actually use the room. It’s like  leaving a digital breadcrumb for me, a record of my new relationship to music (I’m learning to play the drums). A lot of friends hang out there too, often playing their own instruments or riffing on the whiteboards I’ve hung about the place. But  I don’t make it a habit to mention the room’s Foursquare doppelganger. It seems a bit … forced. And as far as I know, many of them don’t use the service.

On the same day I created the RSC on Foursquare (and probably because he asked me what I was doing on my phone), one fellow did check in. With some whimsy, he added a tip: “Try the wings.” It’d make you laugh if you’ve ever been there, trust me. Since then, in the past nine months, countless folks have been through the place, but only one other person has checked in.

Anyway, yesterday this well-known VC came by, and we met in the RSC mainly because it was too loud in my home office (construction going on outside). And as he walked in and sat, he put his iPhone down on a nearby table, as did I. I thought about asking him to check in, but….then I forgot. We spoke for an hour or so, reviewing all manner of things in our industry, discussed our business, and on his way he went.

Then I thought to myself – hey, he should have checked into that space. Then there’d be a record of his visit, and that’d be cool. Kind of like a guestbook of sorts.

But…I don’t even know if the fellow uses Foursquare. And he of course had no idea that the Ross Social Club was “lit up” on the service. And I wasn’t sure mentioning it to him wasn’t, well, kind of dorky.

You see all those social instrumentation and nuance problems I’m having?

Anyway, here’s a thought on one way to add just one wrinkle of nuance to location services. While I’m sure at some point in our collective future the concept of a “place” being digitally “alive” and communicable will be commonplace, at the moment, it’s rare and noteworthy.

As a transition between the two, I’d love a feature on Foursquare (or any other location service, er…say Google, or Facebook, or Twitter…) that allows me to send someone who I’ve been with somewhere (like the VC) an invitation to check in post facto. It’d kind of be like saying “Hey, send your phone over to my house. I’ll check you into the Ross Social Club.” The idea is, he didn’t know he could check in (and I forgot to tell him about it), but I can vouch for his presence there. He should get credit on Foursquare for being there (and the great Database of Intentions would get another bit of data), but he’s back in San Francisco now, so there’s no way for him to check in. But if he “sent his phone over” to me, I could do it for him.

Of course he wouldn’t actually send his phone over, the service would verify me as trustworthy and let me check the VC in on his behalf. But it’d add a rare human element to the service, and I for one would see many uses for it. If nothing else, it’d drive more interaction between people around the platform, and isn’t that what we all want anyway?

Just a random thought. OK, on with work…

“We need some angry nerds”

By - November 30, 2011

Jonathan Zittrain has an important op ed up on Harvard’s site, and I hope all of you will go read it. It sums up many of the points that I hit as I write here at Searchblog, and that will enliven my next book What We Hath Wrought. Key points:

Rising numbers of mobile, lightweight, cloud-centric devices don’t merely represent a change in form factor. Rather, we’re seeing an unprecedented shift of power from end users and software developers on the one hand, to operating system vendors on the other—and even those who keep their PCs are being swept along. This is a little for the better, and much for the worse…..

…in 2008, Apple announced a software development kit for the iPhone. Third-party developers would be welcome to write software for the phone, in just the way they’d done for years with Windows and Mac OS. With one epic exception: users could install software on a phone only if it was offered through Apple’s iPhone App Store. Developers were to be accredited by Apple, and then each individual app was to be vetted, at first under standards that could be inferred only through what made it through and what didn’t. For example, apps that emulated or even improved on Apple’s own apps weren’t allowed.

The original sin behind the Microsoft case was made much worse. The issue wasn’t whether it would be possible to buy an iPhone without Apple’s Safari browser. It was that no other browser would be permitted…

….Developers can’t duplicate functionality already on offer in the Store. They can’t license their work as Free Software, because those license terms conflict with Apple’s.

The content restrictions are unexplored territory. At the height of Windows’s market dominance, Microsoft had no role in determining what software would and wouldn’t run on its machines, much less whether the content inside that software was to be allowed to see the light of screen…

…tech companies are in the business of approving, one by one, the text, images, and sounds that we are permitted to find and experience on our most common portals to the networked world. Why would we possibly want this to be how the world of ideas works, and why would we think that merely having competing tech companies—each of which is empowered to censor—solves the problem?

This is especially troubling as governments have come to realize that this framework makes their own censorship vastly easier…

…A flowering of innovation and communication was ignited by the rise of the PC and the Web and their generative characteristics. Software was installed one machine at a time, a relationship among myriad software makers and users. Sites could appear anywhere on the Web, a relationship among myriad webmasters and surfers. Now activity is clumping around a handful of portals: two or three OS makers that are in a position to manage all apps (and content within them) in an ongoing way….

….If we allow ourselves to be lulled into satisfaction with walled gardens, we’ll miss out on innovations to which the gardeners object, and we’ll set ourselves up for censorship of code and content that was previously impossible. We need some angry nerds.

I’m not a nerd, quite, but I’m sure angry.

Top Searches of 2011 Start Coming In, and They Remain Vapid

By - November 28, 2011

Bing released its top searches of the year today, continuing the trend of presuming the year ends before December begins (watch for Yahoo and Google’s lists in the next week or so). Once again, the data is utterly uninspiring and shallow. I mean, did we really not know that the US is fascinated with celebrities and iPhones?

This is becoming something of a trope for me, but given all the data to which search giants like Microsoft and Google have access, I’d love to see some real data science being applied – find us the conceptual scoops, the insights, the second and third order trends. Is that too much to ask?

The Problem and the Opportunity Of Mobile Advertising

By - November 18, 2011

I hate to pick on the good folks at TextPlus, because I like the service (and my kids do too). But my family recently had an experience that reminded me how stunted the advertising ecosystem remains in the (relatively) new world of apps.

Quite serendipitously, after that experience I had the pleasure of meeting the CEO of Gogii, the company behind TextPlus. More on what I learned from him in a moment. But first, to the problem.

TextPlus is a star in what I’ve come to call “AppWorld,” that Jobsian funhouse mirror universe of apps (TextPlus also on Android, where I’m told it’s growing faster than in iOS). Parent company Gogii is backed by Very Serious Venture Capitalists like Kleiner Perkins and Matrix Partners. Clearly, this is a horse that smart money is backing.

But smart money cannot an ecosystem fix – at least not so far. With some notable exceptions, the mobile advertising ecosystem has been pretty broken. A few reasons why:

1. Every app maker I have spoken to tells me that they mostly rely on third party ad networks. They then complain that those networks “don’t understand our business” and are not aligned with their core interests. These networks provide limited visibility into future revenues, and the ads they deliver are rarely if ever targeted well.

2. These third party ad networks sometimes pay well, but most of the time, they pay very poorly. If an app is going to make it, they have to depend on Apple’s iTunes store and iAds (which brings with it its own set of gordian policy and business strategy issues) or on their own sales force, which is difficult, as most apps don’t have the scale to build and execute salable ad products.

3. Bad actors and practices often creep into how networks execute campaigns on third party apps.

Which brings me to my experience with TextPlus.

I recently helped my daughter set up her TextPlus account. We downloaded and configured the free version (you can buy an ad-free version for a few bucks, more on that later). After about three minutes, my daughter was happily texting my mobile number. It was a cool experience – my daughter doesn’t have a “real phone” yet, but now she can text me just like her older siblings. She proudly held her iPod Touch up to show me her latest text, then asked me why my phone number didn’t have a name like “Dad.” I thought about it for a second and realized she hadn’t set up any Apple contacts yet. So I opened the Contacts app, put myself in there, and reloaded TextPlus. Sure enough, there was “Dad” in the text stream. My daughter was radiant.

As I was about to hand the iPod back to her, however, the entire screen was taken over by an ad for Cadillac. That was bad enough, but to make matters worse, I couldn’t figure out a way to make the ad go away. I had to wait for 15 seconds or so, till the ad cleared. The moment of delight between my daughter and I was ruined (my unscripted burst of expletives probably didn’t help).

In those 15 seconds, my opinion of both Cadillac and TextPlus dropped precipitously. It wasn’t the actual ad – the creative was pretty good in fact (the image at left is not what I saw, but I didn’t manage to get a screengrab). What angered me was the interruption and the lack of fit – why on earth would Cadillac push an ad to a girl who couldn’t even drive yet?  Given I am pretty deep in this industry and aware of the points I made above, I figured it was entirely possible that both Gogii and Cadillac had no idea this experience was happening to me.

Turns out, I was right. And therein lies one of the big problems (and opportunities) in the mobile advertising world.

Rather than bang out a post about how terrible my experience was (I was tempted), I lobbed a call into Gogii to ask if perhaps the ad was a mistake. Over the years at FM, I have had enough experience with similar issues in the HTML web to know it was certainly possible. As it happened, the CEO of Gogii (based in LA), was in San Francisco the next day. We set a meeting.

Scott Lahman is a mobile and gaming veteran, having held senior positions at Activision, EA, JamDat, and now as founder of Gogii. He came by the FM offices and within a few minutes we cleared up the Cadillac ad question – a third party network had “snuck” the ad past TextPlus’s filters. Lahman hates when that happens, but … it happens. He was aware of it (other customers had complained) and he had already addressed the problem.

With that settled, we could dig into the bigger issues of the mobile advertising landscape, from the point of view of a Serious App Maker like Gogii. Gogii has more than 25 million downloads, a very engaged and attractive base of customers (millions of teens), and has begun to build out a content driven social element to its service. It also has some big things planned that I’m not allowed to talk about yet.

While Gogii sells an ad-free version of its app, its future is deeply tied to advertising. This initially seems counter intuitive given you can buy your way out of ads on TextPlus for a few bucks – for life. That’s a pretty low estimate of the lifetime value of a customer. Clearly, something else is going on. At the moment, Gogii has one product that it sells at a premium – essentially a home page skin – but it makes the rest of its revenue from the networks.

Given all that, I wondered how Lanham felt about the three points I made earlier in this piece.

In short, Lanham told me, the mobile advertising industry is about five years behind the HTML web in just about every way. Which is interesting when you think about what was going on in web advertising back in 2006.

Back then, FM was one years old, but our model was gathering steam. Successful websites (the TextPluses of the world) were growing like crazy, but struggling with poor ad network revenue streams. There was a glut of inventory in both traditional media sites as well as “the Independent Web” – sites like Boing Boing, Dooce, NotCot, etc. – but there was no scaleable way to bring truly engaged marketing into the picture. Of course, since then the FM model and many like it have evolved to bring real revenue and relationships to the ecosystem. We’re not where we want to be, of course, but we’ve come a long way.

But at the same time as this progress was beginning, that same glut of inventory was feeding a new set of players – the second-generation ad networks. It was about five years ago that those networks were snapped up in a frenzy of M&A. AOL already had purchased Advertising.com. Google bought DoubleClick (and its nascent exchange business). Yahoo bought Right Media. Microsoft bought AdECN. There was a lot of froth in the marketplace, as the major players realized Internet advertising was ripe for consolidation and new business models.

Since then, we’ve seen the rise of exchanges, demand side platforms, and programmatic buying. We’ve also witnessed the attendant response from the publishing world – supply side platforms like Lijit, hand wringing from premium publishers like ESPN, and innovations in ad products like conversation targeting, which can’t be bought on exchanges (at least, not yet.) We’ve seen online advertising split into two large buckets: “Unsold” or remnant inventory, where the networks mainly play, and “premium,” where publishers (or their partners) create more engaging executions that resonate for brands.

Put another way, a lot has happened in the non-mobile web, and it’s all about to happen again, but faster, in the mobile web. There’ll be a lot of consolidation (the firing shots were, of course, AdMob/Google and Quattro/Apple), there’ll be engaging new products (think really new) that scale across federations of quality apps like TextPlus, and there will be new technology solutions that will, more likely than not, be snapped up for prices that don’t seem to make a ton of sense.

Which is all a long way of saying that I think things are about to get far more interesting in mobile advertising, in particular for apps that comprise “the Independent Web” of the mobile world. It’s about time. In a future post I hope to delve into some of the architecture, data, and business model issues that will need to be addressed if we really want to see a truly independent world of apps thrive. I for one certainly do.