(image) **Updated at 3 PM PST with more info about Facebook/Google negotiations…please read to the bottom…**
In today’s business climate, it’s not normal for corporations to cooperate with each other when it comes to sharing core assets. In fact, it’s rather unusual. Even when businesses do share, it’s usually for some ulterior motive, a laying of groundwork for future chess moves which insure eventual domination over the competition.
Such is the way of business, particularly at the highest and largest levels, such as those now inhabited by top Internet players.
Allow me to posit that this philosophy is going to change over the next few decades, and further, indulge me as I try to apply a new approach to a very present case study: That of Google, Facebook, and Twitter as it relates to Google’s search index and the two social services’ valuable social interaction datasets.
This may take a while, and I will most likely get a fair bit wrong. But it seems worth a shot, so if you feel like settling in for some Thinking Out Loud, please come along.
First, some abridged background. Back in 2009, on the Web 2 Summit stage of all places (yes, I was the emcee), Google, Microsoft, Facebook and Twitter announced a flurry of deals, some of which were worked out in a last minute fury of negotiations. Early in the conference Microsoft announced it would incorporate Twitter and Facebook feeds into its new search engine Bing. Not to be outdone, Google announced a deal with Twitter the next day. However, Google did not announce a deal with Facebook, and the two companies have never come to terms. Meanwhile, Microsoft has continued to deepen its relationship with Facebook data, to the point of viewing that relationship as a key differentiator between Bing and Google search.
All of these deals have business terms, some of them financial, all with limits on how data is used and presented, I would presume. Marissa Mayer of Google told me on the Web 2 stage that there were “financial terms” in Google’s deal with Twitter, but would not give me any details (nor should she have, frankly).
Fast forward to the middle of last year, when the Google/Twitter deal was set to expire. At about the same time as renewal was being negotiated, Google launched Google+, a clear Facebook and Twitter competitor. For reasons that seem in dispute (Google said yesterday Twitter walked away, Twitter has not made a public statement about why things fell apart), the renewal never happened.
And then yesterday, Google incorporated Google+ results into its main search index, sparking a debate in the blogosphere that rages on today – Is Google acting like a monopolist? Does Facebook or Twitter have a “right” to be included in Google results? Why didn’t Google try to negotiate inclusion with its rivals prior to making such a clearly self-serving move?
Google execs, including Chair Eric Schmidt, told SEL’s Danny Sullivan that the company would be happy to talk to both companies to figure out ways to incorporate Twitter and Facebook into Google search, but clearly, those talks could have happened prior to the G+ launch, and they didn’t (or they did, and did not work out – I honestly have no idea). When Danny pointed out that Twitter pages are publicly available, Schmidt demurred, saying that Google prefers to “have a conversation” with a company before using its pages in such a wholesale fashion (er, so did they have one, or not? Anyway…). He has a point (commercial deals are de-rigueur), but…that conversation happened last year, and apparently ended without a deal. And around we go…
What’s clear is this: All the companies involved in this great data spat are acting in what they believe to be their own self interest, and the greatest potential loser, at least in the short term, is the search consumer, who will not be seeing “all the world’s information” but rather “that information which is readily available to Google on terms Google prefers.”
The key to that last sentence is the phrase “what they believe to be their own self interest.” Because I think there’s an argument that, in fact, their true self interest is to open up and share with each other.
Am I nuts? Perhaps. But indulge my insanity for a bit.
The Cost of Blinkered Competition
Back in the Web 1.0 days, when I was running The Industry Standard, I had a number of strong competitors. It’s probably fair to say we didn’t like each other much – we competed daily for news stories, advertiser dollars, and the loyalty of readers. The market for information about the tech industry was limited – there were only so many people interested in our products, and only so much time in the day for them to engage with us.
My strategy to win was clear: We’d make the best product, have the best people, and we’d win on quality. When I heard about one of our competitors badmouthing us, I’d try to ignore it – we were winning anyway: We had the dominant marketshare, the most revenues ($120mm in 2000, with $21mm in EBIDTA), and the best product.
Then something strange happened: an emissary from a competitor called and asked for a meeting. Intrigued, I took it, and was surprised by his offer: Let’s put our two companies together. Apart, he argued, we were simply tearing each other down. Together, we could consolidate the market and insure a long term win.
I considered his idea, but for various reasons, we didn’t take him up on it. I felt like we had the dominant position, that his offer was driven by weakness, not intellectual soundness, and I also felt that a combination would require that my shareholders take on too much dilution.
Two years later, both of us were out of business.
Now, I’m not sure it would have mattered, given the great crash of 2001. But what is certainly true is that I could have thought a bit deeper about what this fellow was proposing. Back in the days of print-bound information, we were essentially competing on what were publicly available assets: stories, particularly interpretations and reportage around those stories, and people: writers, editors, ad sales executives, and management. Short of combining companies, there wasn’t really any other way for us to collaborate, or at least, so I thought.
But perhaps there could have been. It’s been more than a decade since that meeting, and I still wonder: perhaps we could have shared back-end resources like operations, publishing contracts, etc. and saved tens of millions of dollars. We’d compete just on how we leveraged those public assets (stories, people). Perhaps we might have survived the wipeout of the dot com crash. We’ll never know. Since those publications died, the blogosphere has claimed the market, and now it’s far larger than the one we lost back in 2001. Of course I started Federated Media to participate in that model, and now FM has as large a revenue run rate as the Industry Standard, across a far more diverse market.
Why am I bringing this up? Because I think there’s a win-win in this whole Google/Facebook/Twitter dust up, but it’s going to take some Thinking Differently to make it happen.
Imagine Twitter and Facebook offer efficient access to all of their “public” pages – those that its users are happy to share with anyone (or even just to their pre-defined “circles”) – to Google under some set of reasonable usage terms. Financial terms would be minimal – perhaps just enough to cover the costs of serving such a large firehose of data to the search giant. Imagine further that Google, in return, agrees to incorporate this user data in a fashion that is fair – ie doesn’t favor any service over any other – be it Twitter, Google+, or Facebook.
Now, negotiating what is “fair” will be complicated, and honestly, should be subject to iteration as all parties learn usage patterns. And of course all this should be subject to consumer control – if I want to see only Twitter or Facebook or Google+ results in particular searches (or all results for that matter), I should have that right.
And this leads me to my point. Such a set up, regardless of how painful it might be to get right, would create a shared class of assets that would have to compete at the level of the consumer. In other words, the best service for the query wins.
That’s always been Google’s stated philosophy: the best answer for the question at hand. Danny gets to this point in a piece posted last night (which I just saw as I was writing this): Search Engines Should Be Like Santa From “Miracle On 34th Street”. In it he argues that Google’s great strength has been its pattern of sending people to its competitors. And he upbraids Google for violating that principle with its Google+ integration.
It doesn’t have to be this way. It’s not only Google that’s at fault here. Facebook won’t share with Google on any terms, Facebook and Google have not been able to come to terms on how to share data (more on that below*), and Twitter clearly wants some kind of value if it is to share its complete firehose with the search giant. Imagine if all three were to agree on minimal terms, creating a public commons of social data. Yes, that would put Google in an extreme position of trust (not to mention imperil its toddler Google+ service), but covenants can be put in place that allow parties to terminate sharing for clear breaches which demonstrate one party favoring itself over others.
Were such a public commons to be created, then the real competition could start: at the level of how each service interprets that data, and adds value to it in various ways.
Four years ago to the month, I wrote this post: It’s Time For Services on The Web to Compete On More Than Data
In it I said: It’s time that services on the web compete on more than just the data they aggregate….
I think in the end, Facebook will win based on the services it provides for that data. Set the data free, and it will come back to roost wherever it’s best used. And if Facebook doesn’t win that race, well, it’ll lose over time anyway. Such a move is entirely in line with the company’s nascent philosophy, and would be a massively popular move within the ouroborosphere (my name for all things Techmeme).
Compete on service, Facebook, it’s where the world is headed anyway!
Two and a half years ago, as it became clear Facebook’s “nascent philosophy” had changed (and as Twitter rose in stature), I followed up with this post: Google v. Facebook? What We Learn from Twitter. In that post, I said:
I think it’s a major strategic mistake to not offer (Facebook’s pages and social graph) to Google (and anyone else that wants to crawl it.) In fact, I’d argue that the right thing to do is to make just about everything possible available to Google to crawl, then sit back and watch while Google struggles with whether or not to “organize it and make it universally available.” A regular damned if you do, damned if you don’t scenario, that….
For an example of what I mean, look no further than Twitter. That service makes every single tweet available as a crawlable resource. And Google certainly is crawling Twitter pages, but the key thing to watch is whether the service is surfacing “superfresh” results when the query merits it. So far, the answer is a definitive NO.
Why?
Well, perhaps I’m being cynical, but I think it’s because Google doesn’t want to push massive value and traffic to Twitter without a business deal in place where it gets to monetize those real time results.
Is that “organizing the world’s information and making it universally available?” Well, no. At least, not yet.
By making all its information available to Google’s crawlers (and fixing its terrible URL structure in the process), Facebook could shine an awfully bright light on this interesting conflict (of) interest.
Bernes-Lee has a nice critique of why Facebook was bad for the web quite awhile ago that described how their decision to be closed and unfederated network:
http://www.scientificamerican.com/article.cfm?id=long-live-the-web Amit Singhal appears to have advocated for Search Plus Your World given Facebook’s closed business model:http://arstechnica.com/web/news/2011/06/inside-google-how-the-search-giant-plans-to-go-social.ars/2As Eben Moglen has stated several times, the existence of these closed networks are the users fault for not solving the public good dilemma and using open services (of which there are many excellent and fully functional alternatives), and I certainly agree with him.From the outside, it appears that Google’s moves with Google+ are in fact pro-consumer; they are forcing the closed data silo’s that contain our own information to open up by providing a competing service with a competitive feature and at the same time provided data liberation services for our data in their network:
http://www.dataliberation.org/
I’m struggling to listen to the debates in the blogosphere about this issue; they do not appear to integrate the current and predicted consequences of open and closed communication and data systems, of behavioral modification and control by these systems with Google’s apparent business moves. How else is Google to convince social networks to open up without presenting an economic necessity for them to do so?
This soap opera just appears to be a rerun of the Yelp drama.
Interesting, thanks for that input. Good points, well taken.
You cite 2 Google employees and a Google run site as proof for your POV.
Lame “Guest” Googler
The post above is not by a ”
Lame “Guest” Googler”, as suggested below, but instead by an independent scientist who prefers an open to a closed internet (imagine if emaill was closed). The internet has been by far the most inspiring platform for innovation in the past decade and the existence of closed systems presents barriers to innovation, or at the very least, innovation dictated by the EULA and ToS of specific companies. At one point Google’s business model depended upon the web being open; if data is closed it cannot be crawled. I suggest that many aspect of Google’s business models still do but that should not begrudge the closed and unfederated Google+.
I regret that Google+ is an unfederated (it appears as if Google had lobbied in the Valley for federated social services without success) and closed system, and I would prefer that prior open social initiatives (Google’s and other’s) had gained traction, but the fact is they did not. Given the history of the growth of social networks, Google+ appears to be a temporary business imperative and, to be frank, a welcome alternative to the business practices of Facebook.
In the absence of users adopting open social alternatives (e.g. Diaspora), which I view as impractical and unlikely, it seems only prudent to support Google’s moves with Google+ to level the playing field with other social services and overcome the artificial barriers of network effects.
(It is also worth noting that Google may not be as aggressive as one may hope. Here is a nice critique of the misconceptions that have become popular over the past couple of days concerning the Twitter and Google spat: luigimontanez.com/2012/how-rel-nofollow-works/.).
I have clarifications on this issue (Twitter no follow) which I’ll post soon.
Thanks for this thoughtful comment.
I am really looking forward to that
Twitter’s restrictions don’t only restrict Google – they restrict users from accessing their own content.
They prevent me finding my own Tweets beyond the 3K or so that can be retrieved by API,what I have in backup services, and what Topsy may have indexed and cached of their own accord.
Their internal link structure is a great way to promote accounts leveraging Twitter for parasite hosting.
I am sure this is on their very long roadmap…
If Google is allowed access to Facebook and Twitter data then shouldn’t Yandex, DuckDuckGo and other web search companies. If not, then what are the EU regulators going to say?
The big issue is Google’s predatory business actions which are based on ‘shoot first, dodge the questions later’. The Google “do no evil” mantra has been a perfect PR cover for their questionable behaviour.
Point taken, I’d warrant all reputable folks should have access to an API. However, business terms for major players will have to be clear. Perhaps all operate under those terms.
Are you kidding? “Do no evil” is cited by every half-wit Google critic this days.
Wouldn’t keeping data “within circles of friends in the context of a Google search” require Google to either replicate FB’s social graph, or query Facebook’s API for every search that touched FB data? If so, since Facebook clearly wouldn’t let them replicate the graph, wouldn’t the real deal breaker have been requiring that Google accept Facebook’s TOS (which would have pretty serious implications on Google’s ability to run ads against FB-sourced data)?
Josh you point out real complications which are beyond my current comprehension. I hope we can push this dialog into the public….and have Google and Facebook respond to it….
Google+ is linked to one’s google account. If Google were to crawl Facebook and Twitter accounts among ‘friends’, will that mean one has to provide the login information for other networks to Google? If that is the case then there is lot of integration among various services. At that point I suspect people will just start using Google for everything (that is a winning situation for Google) and why bother with other services.
I think you’d have to be logged in to your other service, that’s how Facebook connect et al works, no?
Storm in a tea-cup? It still remains about data and ‘user intent’ performance marketing across the web. Let me remind you of your very own #3 Prediction for 2012: “The Facebook Ad Network”
“This prediction has to do with Facebook announcing and then launching a web-wide advertising network along the lines of Google’s AdSense.” : http://tiny.cc/y6lsx – http://tiny.cc/yyii6
Oh yes, it’s all related!
.Battle of the intermediaries…For me the framework of the discussion around the issues are upside down…All of the companies involved are making decisions about “Community” created content/data without taking into consideration the views of the “Community”.
All of the decisions of Twiiter,FB, and Google are based on generating revenue from “Community” content that has been siloed …..So interesting to see part of the debate concerning rel=nofollow ….as it has been applied to “Community” content by companies that they have not created or own the content….Maybe one day members of the “Communities” can decided to use or not to use rel=nofollow ….I think it works this way outside of the Silos…..
John,
you are too idealistic, bordering on naive. There is a lot Google can crawl and already does from FB and Twitter. if they want to be relevant, they can slice and dice that data like no one else and present it perfectly when people search for Britney Spears, Fords Motors or Oprah.
They do not want to. They do not want to. They do not want to. They do not want to.
There is no reason Google should share their top secrets with Facebook, Twitter or Bing and vice-versa, a lot can be done without that. The question is this: Does Google want to present the most relevant results or increase their earnings short term? Increase their earnings is the answer. Google growth has practically stopped but in order for Larry to be able to fight his war against everyone Google must grow their earnings still or the stock will go south along with dream of carelessly spending tens of billions in Android or…. Starting in 2009 Google has filled their results page with junk, some more useful than others but junk made of local results, images, out of place videos, Google products, news, blogs and each update more is added. This, coincides with earnings increase (surprise!) and less traffic sent to the sites most users came to see. So Google has given up relevancy for profit for a while but it isn’t enough.
Now they have to do more or Larry will not be the genius CEO he supposedly is: by adding another 3-5 or 10 Google+ posts organic search results will be pushed even lower and odds increase that you will click on a very expensive ad. Until I removed my Ad-block and Ghostery I never knew how bad Gogle has become, the real content is hidden and you have to navigate like a minefield if you want to avoid clicking on an ad or Google “service.”
It’s sustainable, people have started to seriously grumble since Panda but Google is in a position to spend billions on their brand and they have very little competition. It cannot last for too long, but they will keep going until it’s too late, bulldozing everyone else that stands in their way.
They actually want. Twitter so Facebook are the one that don’t…
278 and John,
no doubt that they want all the data FB and Twitter has, just as Bing would want Google’s adword secrets or AMD Intel’s. You are missing the bigger point: Google is the one that makes money by being the most relevant, or so they say, so it should be their loss but they are smiling. They have more than enough Twitter pages and FB pages to mash them up without taking everything Twitter and FB have and slap adds on them. Not to mention that #1 gets 3-5 times more clicks. Get how long it will take Google to send them next to nothing in traffic?
Google wants to be a black hole, you leave only if you click an ad even though everything is based on webmaster content. Of course Google’s gain is site-owner’s
loss and since the impartial Google (joke) controls everything,
guess who’s winning?
John isn’t it time credible people like you start asking serious
questions to Google and look past their carefully rehearsed press
releases?
We don’t know what they want? What does Citibank want when they do certain things with your credit card rates John? Do you need to see an internal memo to figure it out?
Hmmm. I thought I did ask serious questions, Jacques, it’s why I wrote the piece before this one, and this one. I’ll keep asking.
John,
you are a good man but Google has been doing these things for a long time: ads are not different from content for John Doe and non-paid content is pushed so far down it barely receives any clicks. All this from a search engine that claims to do what’s best for users, be unbiased and other crap. No one serious has taken them to task. If they claim to be unbiased, they should be, and ads shoudl clearly separated and marked. Words alone don’t do it, they’re master manipulators. Otherwise it’s fraud.
Check this t*co/Btq7MEY7 (replace the *) John and see in what sad state e-commerce pages are and how it effects the ecosystem, small businesses, jobs, competition and our economy. A bully has taken over, pay or no play. Google is getting attention because now it wants to starve everyone of traffic (especially news and tech sites that are popular in G+), but they have been doing it to group after group for many years.
The 13 year honeymoon with the press should be over.
I’m not arguing with what Google’s doing. I am however going to argue (will take time to write) that it was inevitable.
Inevitable? Once trust goes, Google is in a total different ball game. Just wait until the big sites and blogs start to lose traffic (search is almost a zero sum game now) to G+.
Like a petty thief “moving up” and stealing from the mob, with the loot and attention come the consequences and you can lose a lot more. Anti-trust takes years, but members of Congress can push FTC to cripple Google. And I mean legitimately and legally cripple them.
Hint: Look at how ads were labeled a few years ago and how are (not) labeled today. What’s that good for 40%, 50% or more of Google’s earnings gone in one shot? Lots of average people don’t know they’re clicking on ads and Google has A/B tested them for maximum confusion. Make that fraud since it crosses a line.
They may not want to (I don’t think any of us know “What Google Wants”), but what they say is interesting. And what they say in the coming few days/weeks on this issue will be very interesting.
I can tell you right now what they say : “”Our mission is to have the most useful….blah blah blah……we are a different type of company…ethics…”
Say and do are two different things, especially for Google. No corporation should be trusted but Google is on the league of their own even there: betraying everyone, trying to drive all out of business and of course stealing from everyone. Shoudl we mention who Google has picked fight with? John, if you ever meet Larry, Sergey or a Google exec, check your wallet and count your fingers :). They can never have enough.
A few things I was told to observe about Google:
Are ads more relevant than the
search results below?
What % of the sites on page one also have
Google ads? Are ads truly distinguishable from the content for
average users and devices?
How far below are the site listing pushed
by ads and other Google stuff that offer you they joy of clicking on ads once you go there?
Not surprisingly, I was shocked. It’s just too much to be a
coincidence, especially when their earnings grow almost exclusively
based on an increase of adclicks and advertising so they favor advertisers in general. More advertisers means higher prices and more ads in more keywords. Google makes their money when
average users click on ads; they lose when visitors click on
Amazon-com vs an Amazon-com Adwords ad. Once you start from there
and you will understand everything, no matter what Google says.
Google that controls so much is gamed, screwing everyone but themselves. Their “growth” is based on fraud, it’s not organic growth.
before going on an anti-google rant, do you remember the facebook PR disaster for Google. When Google presented the public Facebook data in whatever fashion it conceived, facebook actually paid a shady PR to plant negative stories about this. And when confronted with the truth, facebook said Google was not honoring the terms and conditions of facebook and hence we resorted to this. Yes no company is clean, but Google is comparatively cleaner than Facebook and twitter is acting in a stupid way. Twitter and Google are natural allies. There is no place for Google and Twitter in a facebook dominated world. Please negotiate with Google, twitter and come to some mutually beneficial agreement instead of this media drivel and media posturing.
My understanding is that Google is using public data from Google+ that is available to all, but then displaying the data from that that is most relevant.
Thinking back, did Facebook really have much “public to all” back in 2009?
Others have highlighted API issues to access the graph
I have seen lots of people talk about monopoly but no mention of another evil.
If competitors in a free market all get into bed with each other thus reducing competition, and maybe excluding lesser players, that could be looked on as a cartel.
Oh I think the data should be shared with everyone under broad license and terms.
I don’t think you’re nuts at all.
Perhaps the primary reason for Google’s success in building and consolidating the commanding market share they possess is their historically single-minded obsession with the quality of their search results. They’ve demonstrated time and time again that they know very well where their bread is buttered, and have gone to extraordinary lengths to re-jig their algorithm when it has become apparent those results were lacking (Panda being the latest and greatest example).
The outcry aside, Google’s Plus-enhanced search results are interesting as they relate to two issues concerning search result quality.
First, as your post addresses in length, quality as defined (at least in part) by comprehensiveness is certainly lacking in these new results insofar as they conspicuously omit Twitter and Facebook results. Google’s enhanced results are results enhanced with activity from social networks, but only from a specific subset of available networks. This is of course detrimental for Twitter and Facebook as well, as they are not the beneficiaries of traffic and exposure that would result if they were fully integrated into Google’s results (as an aside I think that Facebook’s need to build a post permalinking structure after the fact, and one that is still somewhat unsatisfactory, demonstrates the shortsightedness of building this sort of walled garden – a corner Google was careful not paint itself into when they created Google+). So yes, the main players have more to gain by cooperating than they do by engaging in battle.
Were I more prone to look for conspiracies, I could almost believe this step by Google to force Twitter and Facebook’s hands, though this is more likely a self-promotional misfire along the line that Danny Sullivan suggests. Whether an intended consequence or not the wheels are already turning, as your update about Facebook demonstrates.
The second issue surrounding the quality of search results the Google move has brought to the fore is the much bigger issue of the appropriateness and value of socially-derived content in the search results in general. A collateral benefit (at least in my eyes) of Google’s move is that it has opened up a rich and necessary discussion surrounding the integration of socially-derived data into search results, picking up where the more loosely-framed discussion around “personalization” left off. To which degree should we rely on the opinions of our friends and acquaintances, weighed against both the collective wisdom of aggregated individual opinions and the seeming authority of institutional sources? Are search providers actually diminishing the power of their own engines to provide useful information and facilitate information discovery by bowing to the desire of users to consume more “socialized” search results? Or are the assumptions being made about what searchers want actually wrong, and are many searchers now feeling alienated because too much weight is being put on social in search results (there’s been lots of chatter along these lines since the Google change)?
I welcome this discussion because it previously has been blithely ignored in the race to integrate more and more social features into search results. The many declarations that search is dead (that is, that social recommendations will eventually replace “searching”) are rooted in an largely unarticulated assumption that information from one’s social circles is superior to other types of information. Google’s move has now very forced an examination of that assumption, and the conversation surrounding it is now well under way.
Very thoughtful Aaron and takes the discussion to places I find really interesting. Thanks.
Microsoft has a stock in Facebook.. Reason to believe how they agreed on FB’s terms.
TYCLLJTMSHBCO
Welcome to Oakley Sunglasses Hut to buy cool and cheap oakleys sunglasses.
John writes: “I considered his idea, but for various reasons, we didn’t take him up on it. I felt like we had the dominant position, that his offer was driven by weakness, not intellectual soundness, and I also felt that a combination would require that my shareholders take on too much dilution.
Two years later, both of us were out of business.”
Congratulations on discovering belatedly that you left money on the table because you weren’t a sophisticated bargainer.
For almost 50 years, since the early 1960’s, academics have been writing about how to manage the tension between creating/claiming value. In the 80’s, the program got a practical boost with Lax and Sebinuis’s “The Manager as Negotiator”, and PON continues to run Creating/Claiming value seminars.
However, I don’t see any of the Google/Facebook crowd having the intellectual rigor and honesty to learn and expect that they will wreck their businesses on shoals of anti-trust lawsuit. Only Hal Varian might be able to persuade some of the engineers at Google that there is a valuable engineering component to game theory inspired collaboration, the type discussed by in Co-opetition.
(Sorry this is so long – didn’t have time to make it shorter.)
No, thank you for the thoughtful comment.
I’ll have more on this subject soon.
That is a kind response. You are welcome.