free html hit counter Future of Search Archives - Page 3 of 8 - John Battelle's Search Blog

On Mayer, Yahoo!, and The (Other) Customer

By - July 18, 2012

Mayer at the Web 2 Summit, San Francisco

(image James Duncan Davidson)

I try to let big news percolate for a few days before weighing in, and it seems even more appropriate to follow that playbook when it came to the scrum around Marissa Mayer joining Yahoo.

Yes, I’ve known both Marissa Mayer (and Ross Levinsohn) professionally, for more than a decade, but so do many other folks, and it seems nearly all of them – Steven Levy and Kara Swisher intelligently among them – have weighed in, multiple times, on what this all means. If you want a rundown, just search for “Marissa Mayer” in Google News.

The coverage has taken its usual course from “Holy Shit!” to “What Will Happen to Ross?” to “Wait, Is Mayer Right for the Job” to “Here’s Our Advice/The Things That Need  to Be Fixed/What Mayer’s Focus Should Be” types of pieces.

This won’t really be any of those. Instead, I find myself thinking about the things I’ve not really seen much coverage of, at least in depth. And true to what I’ve spent a fair amount of time thinking about, they all come down to the intersection of media and technology, and the role marketing plays in that landscape.

When I spoke to Mayer after she was named CEO, I asked the question, almost as a joke – “So is Yahoo! a media or a technology company?” She was quick to respond that she just does not get the debate – of course it’s both. What matters, she pressed, is creating great products that surprise and delight Yahoo! customers.

I couldn’t agree more, yet there is an important nuance here – just who *are* Yahoo’s customers?

Let me step back here and posit something that might upset more than a few of you: Yahoo has two sets of customers, and of course the “end user” is one of them. But the other is the marketer.  And media companies – or “tech companies driven by media revenues,” or however else one might want to phrase it – sometimes ignore this fact at their peril.

I’ll let those of you who find such a statement anathema go ahead and click away – here’s a nice unicorn chaser if you’d like – or you can flame me in the comments (I do respond to most, as long as they’re in English and don’t employ more than the occasional insult).

But those of you who’ve continued to read probably know that I believe, deeply, that commercial publishing is a conversation between three key parties: The reader (or viewer), the publisher/content creator, and the marketer. And while it’s generally been true that this conversation has been all kinds of broken during much of the web’s history, the truth is, it needn’t be that way. Six years ago (!) I wrote a series of posts describing the rise of conversational media and imploring that marketers learn to join the conversation. I think it’s fair to say that this is happening, at scale.

Beyond the contributions of pioneers like Federated Media (yes, I had to plug us), the rise of “native” advertising formats is proof of this. Twitter’s promoted suite is one growing example, as is Facebook’s Sponsored Stories (and its attendant focus on getting brands to be true publishers on the Facebook platform). Pinterest, WordPress (in partnership with FM), and Tumblr are hard at work on “native” solutions for their services as well. All of these advertising solutions pale, however, in comparison to the original “native” advertising format of the Web: Google AdWords.

Many have pointed out that Mayer’s principle weakness, when compared to Levinsohn, is her lack of traditional media and marketing chops. I can say from very deep experience that the marketing business is very much a relationship business – CMOs and agency leaders live in a world driven by ideas, creative and content – and they want to know the people who they do business with, and trust them in a way that is difficult to model algorithmically. Mayer’s detractors point out that she’s not spent much time wooing Madison Avenue, or dealing with the inevitable headaches born of the complex, people-driven businesses that are agencies, marketing clients, and content partners.

While there is some truth in this criticism, I think it overlooks a few things. First and foremost, Mayer is a very fast study, and she already knows how important the traditional media business is to Yahoo. Hell, a quick overview of the company’s financials bears this out, as does a visit to any of its properties, which are dominated by advertising. Yahoo may have a lot of technology behind the covers, but its products are nearly all media products – content intended to gather an audience and provide a place for marketers to message to that audience. More than half of Yahoo’s revenues come from “display” advertising, most of the rest comes from search, which is also marketer driven.

Secondly, Mayer will be a big draw of talent, and not just engineering talent. She understands that if she can’t retain Levinsohn and/or his recent CRO Michael Barrett (I certainly hope she can), she’ll need to attract top tier media minds to the business. And I think she’ll succeed at doing just that.

But to me, the thing many are missing is that Mayer will bring her fanatical product focus to more than just Yahoo’s consumer-facing media offerings. She’ll also be staring at the company’s advertising products, and asking this simple question: How can we do better?

To answer that question, Mayer will need to do more than study the data (though of course, that will be important). She’ll need to sit down with a wide swath of Yahoo’s marketing customers and ask them what they want from their investment in her platform. She’ll hear an awful lot of conflicting advice, but it’s in the bricollage from all the feedback that the best ideas come out. Mayer can’t afford to immediately tack away from all those boxes and rectangles cluttering up the Yahoo! experience, nor should she – it turns out that display advertising does indeed work for marketers. But the larger question remains: Can we do better?

The answer lies in executing the subtle and ongoing iterative work of true digital publisher – improving the core product experience both sets of customers – consumers of the media experience, as well as marketers looking to be part of that experience in a more native fashion. And again, from a quick study of Yahoo’s products, there’s plenty of improvements to be made.

An important and related part of the work ahead for Mayer and her team will be deciding what role ad tech and search will play in Yahoo’s future. Despite purchasing Right Media back in 2007, Yahoo has never been seen as a leader in ad tech, and word on the street in the weeks prior to Mayer’s ascension was that Yahoo was about to outsource its ad technology platform to market leader Google. Of course, such a move is fraught with regulatory and business implications. And Mayer may well decide it’s in Yahoo’s best interest to invest in own its own destiny when it comes to the machine-driven world of ad serving and programmatic audience buying. But trust me, what Yahoo does here will be an extremely important directional indicator.

Which brings us to search. It’s been widely reported that Yahoo’s 2009 deal to outsource core search to Microsoft hasn’t worked out as well as either party wished it would. Given how important search is to Yahoo overall, and how deeply knowledgeable Mayer is in this particular field, I’d expect big changes in Yahoo Search. The company recently unveiled a new search product called “Axis,” which seems like a neat idea but feels a bit too complicated for most consumers to really grok. Mayer will likely take Occam’s Razor to search, and I expect the results will be quite positive.

But it’s the other side of Yahoo’s revenue equation – the branded display market – where Mayer will face her greatest challenges, and find her biggest opportunities. Yahoo isn’t a startup like Pinterest, Tumblr,  or even Twitter, where founders can leverage massive user growth to raise enough capital to “figure out how best to implement appropriate native marketing solutions.” Yahoo is nearly 20 years old, and it’s got a very deep, tangled, and somewhat tarnished brand in the minds of its best advertising customers. It’s true that creating world-beating consumer-facing products will go a long way toward fixing that brand. But those products must be informed by – and even created for – both sets of customers – the consumers of content, as well as those who pay for them to be created in the first place.

  • Content Marquee

Larry Page Makes His Case

By - April 05, 2012

Given the headlines, questions, and legal actions Google has faced recently, many folks, including myself, have been wondering when Google’s CEO Larry Page would take a more public stance in outlining his vision for the company.

Well, today marks a shift of sorts, with the publication of a lenthy blog post from Larry titled, quite uninterestingly, 2012 Update from the CEO.

I’ve spent the past two days at Amazon and Microsoft, two Google competitors (and partners), and am just wrapping up a last meeting. I hope to read Page’s post closely and give you some analysis as soon as I can. Meanwhile, a few top line thoughts and points:

– Page pushes Google+ as a success, citing more than 100 million users, but still doesn’t address the question of whether the service is truly being used organically, rather than as a byproduct of interactions with other Google products. I’m not sure it matters, but it’s a question many have raised. He also doesn’t address, directly, the tempest over the integration of G+ into search.

– Page also does not directly address the issue of FTC privacy investigations into the company, not surprising, given any company’s response to these investigations is usually “no comment.” However, Google might have explained with a bit more gusto the reasons for its recent changes.

– Page tosses out another big number, this one around Android: 850K activations a day. Take that, Apple!

– Page uses the words “love” and “beauty” – which I find both refreshing and odd.

– Page also talks about making big bets, focusing on fewer products, and how it’s OK to not be exactly sure how big bets are going to make money. This is a topic where Google has a ton of experience, to be sure.

More when I get out of my last meeting….

If-Then and Antiquities of the Future

By - April 03, 2012

Over the past few months I’ve been developing a framework for the book I’ve been working on, and while I’ve been pretty quiet about the work, it’s time to lay it out and get some responses from you, the folks I most trust with keeping me on track.

I’ll admit the idea of putting all this out here makes me nervous – I’ve only discussed this with a few dozen folks, and now I’m going public with what I’ll admit is an unbaked cake. Anyone can criticize it now, (or, I suppose, steal it), but then again, I did the very same thing with the core idea in my last book (The Database of Intentions, back in 2003), and that worked out just fine.

So here we go. The original promise of my next book is pretty simple: I’m trying to paint a picture of the kind of digital world we’ll likely live in one generation from now, based on a survey of where we are presently as a digital society. In a way, it’s a continuation and expansion of The Search – the database of intentions has expanded from search to nearly every corner of our world – we now live our lives leveraged over digital platforms and data. So what might that look like thirty years hence?

As the announcement last year stated:

WHAT WE HATH WROUGHT will give us a forecast of the interconnected world in 2040, then work backwards to explain how the personal, economic, political, and technological strands of this human narrative have evolved from the pivotal moment in which we find ourselves now.

That’s a pretty tall order. At first, I spent a lot of time trying to boil any number of oceans – figuring out who to talk to in politics, energy, healthcare, technology, and, well, just about every major field. It quickly became quite evident that I’d end up with a book a thousand miles wide and one inch deep – unless I got very lucky and stumbled upon a perfect narrative actor that tied it all up into one neat story. Last time Google provided me that actor, but given I’m writing a book about how the world might look in 30 years, I’m not holding my breath waiting for another perfect protagonist to step out a time machine somewhere.

But what if those protagonists are already here? Allow me to explain…

For the past few months I’ve been stewing on how the hell to actually write this book I’ve promised everyone I would deliver. The manuscript is not actually due till early next year, but still, books take a lot of time. And every day that goes by without a clear framework is a day partially lost.

A couple of months ago, worried that I’d never figure this thing out (but knowing there had to be a way), I invited one of  my favorite authors (and new Marin resident) Steven Johnson over to my house for a brainstorming session. I outlined where I was in my thinking, and posed to him my essential problem: I was trying to do too much, and needed to focus my work on a narrative that paid off the promise, but didn’t read like a textbook, or worse yet, like a piece of futurism. As I said to Steven, “If I write a book that has a scene where an alarm clock wakes you up on a ‘typical morning in 2045,’ please shoot me.”

It’s not that I don’t appreciate futurism – it’s just that I truly believe, as William Gibson famously put it, that the future is already here, it’s just unevenly distributed. If I could just figure out a way to report on that future, to apply the tools of journalism to the story of the future we’re creating, I’d come up with a book worth reading. Of course, it was this approach we took in the early years of Wired magazine. Our job, as my colleague Kevin Kelly put it, was to send writers off in search of where the future was erupting, with instructions to report back.

To find that future, we asked our writers (and editors) to look hard at the present, and find people, places or things that augured what might come next. Hence, issue one of Wired had articles about the future of war, education, entertainment, and sex, based on reporting done in the here and now. While we didn’t call it such, over the years we developed an “If-Then” approach to many of the stories we’d assign. We’d think out loud: “If every school had access to the Internet, then what might change about education?” Or, “If the government had the ability to track everything we do both offline and on, then what might our society look like?” The conditional “If” question followed with a realistic “Then” answer provided a good way to wrap our heads around a sometimes challenging subject  (and for you programmers out there, we’d also consider the “ands” as well as the “elses.”)

Next, we’d ask a reporter to go find out all he or she could about that scenario – to go in search of artifacts from the future which told a story of where things might be going. (Wired, in fact, later created the popular “Found: Artifacts from the Future” series in the pages of the magazine.)

As an early reader and contributor to Wired, Steven knew all this, and reminded me of it as we spoke that day at my house. What if, he asked me, the book was framed as a series of stories about “future antiquities” or “future relics” (I think he first dubbed them “Magic Coins”)? Could we find examples of things currently extant, which, if widely adopted over the next generation, would presage significant changes in the world we’ll be inhabiting? Why, indeed, yes we could. Immediately I thought of five or six, and since that day, many more have come to mind.

Now, I think it bears unpacking this concept of what I mean by “widely adopted.” To me, it means clearing a pretty high hurdle – by 2045 or so, it’d mean that more than a billion people would be regularly interacting with whatever the future antiquity might be.  When you get a very large chunk of the population engaged in a particular behavior, that behavior has the ability to effect real change on our political, social, and cultural norms. Those are the kind of artifacts I’m looking to find.

As a thought experiment, imagine I had given myself this assignment back in the early 1980s, when I was just starting my love affair with this story as a technology reporter (yes, there’s a symmetry here – that’s 30 years ago – one generation past). Had I gone off in search of digital artifacts that presaged the future, ones that I believed might be adopted by a billion or more people, I certainly would have started with the personal computer, which at that point was counted in the high hundreds of thousands in the US. And I also would have picked the Internet, which was being used, at that point, by only thousands of people. I’d have described the power of these two artifacts in the present day, imagined how technological and social change might develop as more and more people used them, and spoken to the early adopters, entrepreneurs, and thinkers of the day about what would happen if a billion or more people were using them on a regular basis.

An antiquity from the 1980s, with its future descendant (image from machinelake.com)

Pushing the hypothetical a bit further, I imagine I’d find the Dan Bricklins, Vint Cerfs, Ray Ozzies, and Bill Gates of the day, and noticed that they hung out in universities, for the most part. I’d have noticed that they used their computers and online networks to communicate with each other, to share information, to search and discover things, and to create communities of interest. It was in those universities where the future was erupting 30 years ago, and had I been paying close attention, it’s plausible I might have declared email, search, and social networks – or at least “communities on the Internet” – as artifacts of our digital future. And of course, I’d have noticed the new gadget just released called the mobile phone, and probably declared that important as well. If more than a billion people had a mobile phone by 2012, I’d have wondered, then what might our world look like?

I’m pretty sure I’d have gotten a lot wrong, but the essential framework – a way to think about finding and declaring the erupting future – seems a worthy endeavor. So I’ve decided to focus my work on doing just that. It helps that it combines two of my favorite approaches to thinking – anthropology and journalism. In essence, I’m going on a dig for future antiquities.

So what might some of today’s artifacts from the future be? I don’t pretend to have an exhaustive list, but I do have a good start. And while the “If-Then” framework could work for all sorts of artifacts, I’m looking for those that “ladder up” to significant societal change. To that end, I’ve begun exploring innovations in energy, finance, health, transportation, communications, commerce – not surprisingly, all subjects to which we have devoted impressive stone buildings in our capital city. (Hence my trip to DC last week.)

Here’s one example that might bring the concept home: The Fitbit. At present, there are about half a million of them in the world, as far as I can tell (I’m meeting with the company soon). But Fitbit-like devices are on the rise – Nike launched its FuelBand earlier this year, for example. And while the first generation of these devices may only appeal to early adopters, with trends in miniaturization, processing power, and data platforms, it’s not hard to imagine a time when billions of us are quantifying our movement, caloric intake and output, sleep patterns, and more, then sharing that data across wide cohorts so as to draw upon the benefits of pattern-recognizing algorithms to help us make better choices about our behavior.

If that were to happen, what then might be the impact on our healthcare systems? Our agricultural practices and policies? Our insurance industries? Our life expectancies? I’m not entirely sure, but it’d sure be fun to try to answer such questions.

I won’t tip my hand as to my entire current list of Future Antiquities, but I certainly would welcome your ideas and input as to what they might be. I’d also like your input on the actual title of the book. “What We Hath Wrought” is a cool title, but perhaps it’s a bit….too heady. Some might even call it overwrought. What if I called the book “If-Then”? I’m thinking about doing just that. Let me know in comments, and as always, thanks for reading.

Apple Gets Into (App) Search

By - February 26, 2012

It took longer than I thought it would, but it’s finally happened. Apple’s admitted that it needs real search to bring it’s tangled app universe to heel, and purchased Chomp, a leading third-party app review and search service.

Nearly two years ago I wrote this piece:  Apple Won’t Build a (Web) Search Engine. From it:

…but it will build the equivalent of an app search engine. It’s crazy not to. In fact, it has to. It already has app discovery via the iTunes store, but it’s terrible, with no signal that gives reliable results based on accrued intent.

What Apple needs is a search engine that “crawls” apps, app content, and app usage data, then surfaces recommendations as well as content . To do this, mobile apps will need to make their content available for Apple to crawl. And why wouldn’t you if you’re Yelp, for example? Or Facebook, for that matter? An index of apps+social signal+app content would be quite compelling.

What Apple will NOT do is crawl the entire web.

I still think that last sentence will remain true. It’s my hope that Apple continues to develop Chomp and help “real search” – the kind I describe above, to happen in iOS.

Thinking Out Loud About Voice Search: What’s the Business Model?

By - February 10, 2012

(image) I don’t have Siri yet – I’m still using my “old” iPhone 4. But I do have my hands on a new (unboxed) Nexus, which has Google Voice Actions on it, and I’m sure at some point I’ll get a iPhone 4GS. So this post isn’t written from experience as much as it’s pure speculation, or as I like to call it, Thinking Out Loud.

But driving into work yesterday I realized how useful voice search is going to be to me, once I’ve got it installed. Stuck in traffic, I tried searching for alternate routes, and it struck me how much easier it’d be to just say “give me alternate routes.” That got me thinking about all manner of things – many of which are now possible – “Text my wife I’ll be late,” “Email my assistant and ask her to print the files for my 11 am meeting,” “Find me a good liquor store within a mile of here,” (I’ve actually done that one using Siri on my way to a friend’s house last weekend).

I’ve written about this before, of course (see Texting Is Stupid, for one example from over three years ago), and I predicted in 2011 that voice was going to be a game changer. It clearly is, but now my question is this: What’s the business model?

I hate to pick on Google, but it’s worth asking the question, given how it dominates mobile search: What happens to the AdWords business model when a large percentage of mobile searches are done using voice? Given we don’t look at our screens while using voice commands (pretty much the whole point, no?), how will Google make money from voice search?

It’s an interesting question, but not for Apple – Apple doesn’t make money through search ads, so it can give voice search away for free, and use it as a benefit of buying and using the hardware device (which is where Apple makes its coin, after all). And from what I can tell, Apple uses Yahoo, Wolfram, Yelp and others to populate Siri’s search answers, not Google. I’m sure there’s a direct reason for that: Google probably wanted some kind of fee from Apple, and I’m guessing Apple had little interest in paying. (I also don’t know if Apple is paying Yahoo, Wolfram or Yelp, if any of you do, please let me know…)

Now, Google does have one model in market that could translate to money in voice search – what it calls “Click to Call.” This is the ability for businesses to integrate direct phone calling into their mobile ads. I don’t know if that model is integreated into Voice Actions, but I’d be surprised if it didn’t show up soon (I can imagine Google’s version of Siri asking “Would you like to call this business now?”). And while that should prove a decent revenue stream, it won’t cover the majority of voice searches. And Google isn’t a company that likes to give away search without a monetization strategy.

What do you think such a strategy might be? Could we even imagine the return of “paid inclusion” – where voice search results are returned based on who pays to be part of the results? Sounds far fetched, but at the right scale, it could work.

I’ve not done much thinking about this, but I bet some of you have. What do you say?

Larry Page’s “Tidal Wave Moment”?

By - February 07, 2012

Who remembers the moment, back in 1995, when Bill Gates wrote his famous Internet Tidal Wave Memo? In it he rallied his entire organization to the cause of the Internet, calling the new platform an existential threat/opportunity for Microsoft’s entire business. In the memo Gates wrote:

“I assign the Internet the highest level of importance. In this memo I want to make clear that our focus on the Internet is crucial to every part of our business. The Internet is the most important single development to come along since the IBM PC was introduced in 1981.”

The memo runs more than 5300 words and includes highly detailed product plans across all of Microsoft. In retrospect, it probably wasn’t a genius move to be so transparent – the memo became public during the US Dept. of Justice action against Microsoft in the late 1990s.

It strikes me that Larry Page at Google could have written such a memo to all Googlers last year. Of course, Page and his advisors must have learned from Microsoft’s mistakes, and certainly don’t want a declarative memo floating around the vast clouds of Internet eternity. Bad things can happen from direct mandates such as those made by Gates – in the memo he mentions that Microsoft must “match and beat” Netscape, for example, words that came back to haunt him during the DOJ action.

Here’s what Page might have written to his staff in 2011, with just a few words shifted:

” I assign social networking the highest level of importance. In this memo I want to make clear that our focus on social networking is crucial to every part of our business. Social networking is the most important single development to come along since Google was introduced in 1998.”

I very much doubt Page wrote anywhere that Google must “match and beat” Facebook. And unlike Gates, he probably did not pen detailed memos about integrating Google+ into all of Google’s products (as Gates did – for pages – declaring that Microsoft must integrate the Internet into all of its core products.)

But it’s certainly not lost on any Googler how important “social” is to the company: all of their bonuses were tied to social last year.

So why am I bringing this up now? Well, I’ve got no news hook. I’m just doing research for the book, and came across the memo, and its tone and urgency struck a familiar note. The furor around Search Plus Your World has died down, but it left a bad taste in a lot of folks’ mouths. But put in the context of “existential threat,” it’s easier to understand why Google did what it did.

Unlike the Internet, which was a freely accessible resource that any company could incorporate into its products and services, to date “social” has been dominated by one company, a company that Google has been unable to work with. Imagine if, when Gates wrote his Tidal Wave memo, the “Internet” he spoke of was controlled entirely by, say, MCI, and that Microsoft was unable to secure a deal to get all that Internet goodness into its future products.

That seems to be where Google finds itself, at least by its own reckoning. To continue being a great search engine, it needs the identity and relationship data found, for the most part, behind Facebook’s walls.

I’ve written elsewhere about the breakdown of the open web, the move toward more “walled gardens of data,” and what that does to Google’s ability to execute its core business of search. And it’s not just social – readers have sent me tons of information that predict how mobile, in particular, will escape the traditional reaches of Google’s spidering business model. I hope to pore through that information and post more here, but for now, it’s worth reading a bit of history to put Google’s moves into broader context.

Our Google+ Conundrum

By - January 14, 2012

I’m going to add another Saturday morning sketch to this site, and offer a caveat to you all: I’ve not bounced this idea off many folks, and the seed of it comes from a source who is unreservedly biased about all this. But I thought this worth airing out, so here you have it.

Given that Google+ results are dominating so many SERPs these days, Google is clearly leveraging its power in search to build up Google+. Unless a majority of people start turning SPYW (Search Plus Your World) off, or decide to search in a logged out way, Google has positioned Google+ as a sort of “mini Internet,” a place where you can find results for a large percentage of your queries.(My source is pretty direct about this: “Google has decided that beating Facebook is worth selling their soul.”)

But to my point. An example of samesaid is the search I did this morning for that Hitler video I posted. Here’s a screenshot of my results:

 

As you can see, the Universal search feature kicked in, and put News results at the top. I know that news results won’t get me straight to the video, I want the YouTube or Vimeo page, not a story about the video. So I look to the results below. The next four results are from Google+. Right below the fold is the actual YouTube video. I didn’t see it on first blush.

So I found that video by clicking on someone’s Google+ post about it (see how the first one is purple, and not blue? That’s the one I clicked on). Some dude I don’t know posted it to Google+, I clicked through to his post (gaining Google another pageview), then clicked through the video to YouTube. That’s lame. That’s not a Googley search experience.

But if that’s how the world of Google works now, that means it’s very important that you tend your Google+ pages, so that you rank well in Google search. Google has pretty much gamed its own search engine to insure Google+ will succeed.

This is what happens when you tell your entire staff that your salary depends on winning in social. 

Now, this presents us all a conundrum. If a large percentage of people are logged into Google and/or Google+ when they are searching for stuff, that means Google+ pages are going to rank well for those people. Hence, I really have no choice but to play Google’s game, and tend to my Google+ page, be I a brand, a person, a small business…. are you getting the picture here? If you decide to NOT play on Google+, you will, in essence, be devalued in Google search, at least for the percentage of people who are logged in whilst using Google.

I dunno. This strikes me as wrong. I’ve spent nearly ten years building this site, Searchblog, and it has tens of thousands of inbound links, six thousand posts, nearly 30,000 comments, etc., etc. But if you are logged into Google+ and search for me, you’re going to get my Google+ profile first.

Seems a bit off. Seems like Google is taking the first click away from me and directing it to a Google service.

Now, if I decide to protest this, and delete my Google+ account, I better pray no one else named John Battelle creates a Google+ account, or they will rank ahead of me. And while Battelle is a pretty unique name, there are actually quite a few of us out there. Imagine if my name was John Kelly? Or Joe Smith?

Yikes. Quite a conundrum.

Again, just sketching on a Saturday morning. It’s a beautiful day, so I think I’ll stop, take a ride, and think a bit more about it before I write anymore.

Related:

It’s Not About Search Anymore, It’s About Deals

Hitler Is Pissed About Google+

Google Responds: No,That’s Not How Facebook Deal Went Down (Oh, And I Say: The Search Paradigm Is Broken)

Compete To Death, or Cooperate to Compete?

Twitter Statement on Google+ Integration with Google Search

Search, Plus Your World, As Long As It’s Our World

 

Google Responds: No,That’s Not How Facebook Deal Went Down (Oh, And I Say: The Search Paradigm Is Broken)

By - January 13, 2012

(image) I’ve just been sent an official response from Google to the updated version of my story posted yesterday (Compete To Death, or Cooperate to Compete?). In that story, I reported about 2009 negotiations over incorporation of Facebook data into Google search. I quoted a source familiar with the negotiations on the Facebook side, who told me  “Senior executives at Google insisted that for technical reasons all information would need to be public and available to all,” and “The only reason Facebook has a Bing integration and not a Google integration is that Bing agreed to terms for protecting user privacy that Google would not.”

I’ve now had conversations with a source familiar with Google’s side of the story, and to say the company disagrees with how Facebook characterized the negotiations is to put it mildly. I’ve also spoken to my Facebook source, who has clarified some nuance as well. To get started, here’s the official, on the record statement, from Rachel Whetstone, SVP Global Communications and Public Affairs:

“We want to set the record straight. In 2009, we were negotiating with Facebook over access to its data, as has been reported.  To claim that the we couldn’t reach an agreement because Google wanted to make private data publicly available is simply untrue.”

My source familiar with Google’s side of the story goes further, and gave me more detail on why the deal went south, at least from Google’s point of view. According to this source, as part of the deal terms Facebook insisted that Google agree to not use publicly available Facebook information to build out a “social service.” The two sides had already agreed that Google would not use Facebook’s firehose (or private) data to build such a service, my source says.

So what does “publicly available” mean? Well, that’d be Facebook pages that any search engine can crawl – information on Facebook that people *want* search engines to know about. This is compared to the firehose data that was the core asset being discussed between the parties. This firehose data is what Google would need in order to surface personal Facebook pages relevant to you in the context of a search query. (So, for example, if you were my friend on Facebook, and you searched for “Battelle soccer” on Google, then with the proposed deal, you’d see pictures of my kids’ soccer games that I had posted to Facebook).

Apparently, Google believed that Facebook’s demand around public information could be interpreted  as applying to how Google’s own search service was delivered, not to mention how it (or other products) might evolve. Interpretation is always where the devil is in these deals. Who’s to say, after all, that Google’s “social search” is not a “social service”? And Google Pages, Maps, etc. – those are arguably social in nature, or will be in the future.

Google balked at this language, and the deal fell apart. My Google source also disputes the claim that Google balked at being able to technically separate public from private data. Conversely, my Facebook source counters that the real issue of public vs. private had to do with Google’s refusal to honor changes in privacy settings over time – for example, if I deleted those soccer pictures, they should also be deleted from Google’s index. There’s a point where this all devolves to she said/he said, because the deal never happened, and to be honest, there are larger points to make.

So let’s start with this: If Facebook indeed demanded that Google not use publicly available Facebook data, it’s certainly understandable why Google wouldn’t agree to the deal. It may not seem obvious, but there is an awful lot of publicly available Facebook pages and data out there. Starbucks, for example, is more than happy to let anyone see its Facebook page, no matter if you’re logged in or not. And then there’s all that Facebook open graph data out on the public web – tons of sites show Facebook status updates, like counts and so on in a public fashion. In short, asking Google to not leverage that data in anything that might constitute a “social service” is anathema to a company who claims its mission to crawl all publicly available information, organize it, and make it available.

It’s one thing to ask that Google not use Facebook’s own social graph and private data to build new social services – after all, the social graph is Facebook’s crown jewels. But it’s quite another thing to ask Google to ignore other public information completely.

From Google’s point of view, Facebook was crippling future products and services that Google might create, which was tantamount to an insurance policy of sorts that Google wouldn’t become a strong competitor, at least not one that  leverages public information from Facebook. Google balked. If Facebook’s demand could have been interpreted as also applying to Google’s search results, well, that’s a stone cold deal killer.

I certainly understand why Facebook might ask for what they did, it’s not crazy. Google might well have responded by narrowing the deal, saying “Fine, you don’t build a search engine, and we won’t build a social network. But we should have the right to create other kinds of social services.” As far as I know, Google didn’t chose to say that. (Microsoft apparently did). And I think I know why: The two companies realized they were dancing on the head of a pin. Search = social, social = search. They couldn’t figure out a way to tease the two apart. Microsoft has cast its lot with Facebook, Google, not so much.

When high stakes deals fall apart, both sides usually claim the other is at fault, and that certainly seems to be the case here. It’s also the case with the Twitter deal, which I’ve gotten a fair amount of new information about as well. I hope to dig into that in another post. For now, I want to pull back a second and comment on what I think is really going on here, at least from the perspective of a longer view.

Our Cherished Search Paradigm Is Broken (But We Will Fix It….Eventually)

I think what we have here is a clear indication that the search paradigm we’ve operated under for a decade or so is broken. That paradigm stems from Google’s original letter to shareholders in 2004. Remember this line?Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating.

In many cases, it’s simply naive to claim Google is unbiased or objective. Google often favors its own properties over others, as Danny points out in Real-Life Examples Of How Google’s “Search Plus” Pushes Google+ Over Relevancy and others have also detailed. But there is a reason: if you’re going to show results from all other possible contenders, replete with their associated UI and functional bells and whistles (as Google does with its own Maps, Pages, Plus etc.), well, it’s nearly impossible now to determine which service is the right answer to a particular person’s query. Not to mention, you need to put a deal in place to get all the functionality of the service. Instead, Google has opted, in many cases, to go with their own stuff.

This is not a new idea, by the way. Yahoo’s been doing it this way from the beginning. The contentious issue is that biasing some results toward Google’s own products runs counter to Google’s founding philosophy.

I have a theory as to why all this is happening, and I don’t entirely blame Google. Back when search wasn’t personalized, Google could defensibly say that one service was better than another because it got more traffic, was linked to more (better PageRank), and so on. Back when everyone got the same results and the web was one homogenous glob of HTML, well, you could claim “this is the best result for the general population.” But personalized search has broken that framework – I lamented this back in 2008 with this post: Search Was Our Social Glue. But That Is Dissolving (more here).

With the rise of Facebook and the app economy, the problem of search has become terribly complicated. If you want to have results from Facebook in your search, well, that search service has to do a deal with Facebook. But what if you want results from your running app (I have hundreds of rides and runs logged on AllSportGPS, for example)? Or Instagram? Or Path, for that matter? Do they all have to do deals with Google and Bing? There are so many unconnected pieces of the Internet now (millions of apps, most of our own Facebook experiences, etc. etc.) that what’s a good personal result for one person is not necessarily good for another. If Google is to stay true to its original mission, it needs a new framework and a massive number of new signals – new glue – to put the pieces back together.

There are several ways to resolve this, and in another post, I hope to explore them (one of them, of course, is simply that everyone should just go through Facebook. That’s the vision of Open Graph). But for now, I’m just going to say this: The issues raised by this kerfuffle are far larger than Google vs. Facebook, or Google vs. Twitter. We are in the midst of a major search paradigm shift, and there will be far more tears before it gets resolved. But resolve it must, and resolve it will.

Whisperings of the Future Surround Us

By - November 17, 2011

Yesterday I met with Christopher Ahlberg, the PhD co-founder of Recorded Future, a company I noted in these pages back in mid-2010. Ahlberg is one of those rare birds you just know is making stuff that matters – a scientist, an entrepreneur, a tinkerer, and an enthusiast all wrapped into one.

He ran me through Recorded Future’s technology and business model, and I found it impressive. In fact, I’m hoping I can employ it somehow into my book research. And that conditional tense of “hoping” is the main problem I have with Ahlberg’s creation – it’s a rather complicated system to use. Then again, what of worth isn’t, I suppose?

Recorded Future is, at its core,  a semantic search engine that consumes tens of thousands of structured information feeds as its “crawl.” It then parses this corpus for several core assets: Entities, Events, and Time (or Dates). Recorded Future’s algorithms are particularly adept at identifying and isolating these items, then correlating them at scale. If that sounds simple, it ain’t.

The service then employs a relatively complicated query structure that allows you to project the road ahead for your question. For example, you might choose “Amazon” as your entity, and then set your timeframe for events involving Amazon over the past two months and into the next two months. Recorded Future will analyze its sources (SEC filings, blogs, news sites, etc) and create a timeline-like “map” of things that have happened and are predicted to happen with regard to Amazon over the next eight weeks. You can further refine a search by adding other entities or events (“earnings” or “CEO”, for example).

How does it work? Well, turns out the Internet is rife with whisperings of the future, you just need to learn how to listen. That’s Recordable Future’s specialty. As you might imagine, Wall Street quants and government spooks just love this stuff. I’d imagine journalists would as well, but most of us are too strapped to afford the company’s services. Embedded below is a new feature of the site, a weekly overview of a news-related entity.

Recorded Future’s engine is not limited to the sources it currently consumes. Not only is Ahlberg adding more every month, his customers can add their own corpuses. Imagine throwing Wikileaks into Recorded Future, for example.

Perhaps the coolest aspect of the service is a visualization of how entities relate to each other over time. Ahlberg showed me a search for mobile patents, then toggled into a relationship graph. Guess what entity broke into the center of the graph, connected to nearly everything else? Yup – Motorola.

Did I mention that Google is an investor in Recorded Future?

As I said, I hope to start using the service soon, and perhaps posting my findings here.