free html hit counter Future of Search Archives - Page 2 of 8 - John Battelle's Search Blog

Search and Immortality

By - September 19, 2013

google.cover.inddFunny thing, there I was two days ago, at Google’s annual conference, watching Larry Page get asked questions so pliant in nature they couldn’t be called softballs. They were more like tee balls – little round interrogatives gingerly placed on a plastic column for Page to swat out into the crowd. Not that we would expect anything else – to be clear, this is Google’s event, and I see nothing wrong with Google scripting its own event. I had moderated the final session of the day, but Larry was the final speaker. Perhaps wisely, Google brought  someone else on to “grill” Page – those were his words as the interview started. (You be the judge –  a sample question: “What are your thoughts about tablets in schools?”)

Anyway, I was certainly not the right choice to talk to Larry. I know the folks at Google well, and have tons of respect for them. We both know I would have insisted on asking about a few things that were, well, in the news at the moment of that interview on Tuesday. Like, for example, the fact that Google, on the very next day, was going to announce the launch of Calico, a company seeking to solve that “moonshot” problem of aging. Oh, and by the way, current Apple Chair and former Genentech CEO Arthur Levinson was going to be CEO, reporting to Page. Seems like pretty interesting news, no? And yet, Larry kept mum about it during the interview. Wow. That’s some serious self control.

And yet I think I understand – each story has its own narrative, and this one needed room to breathe. You don’t want to break it inside an air-conditioned ballroom in front of your most important clients. You want to make sure it gets on the cover of Time (which it did), and that the news gets at least a few days to play through the media’s often tortured hype cycle. It’s grinding its way through that cycle now, and I’m sure we’ll see comparisons to everything from Kurzweil (who now works at Google) to Bladerunner, and beyond.

But what I was reminded of was the very end of my book on search, some 8 years ago. I was trying to put the meaning of search into context, and I found myself returning again and again to the concept of immortality.  This was my epilogue, which I offer here as perhaps some context for Google’s announcement this week:

“Search and Immortality”

On a fine sunny morning in 2003, not long after the birth of my third and most likely final child, I typed “immortality” into Google and hit the “I’m feeling lucky” button. I can’t explain why I turned to a search engine for metaphysical comfort, but I sensed the search might lead me somewhere—here I was writing a book about search, but what did it matter, really, in the larger scheme of things?

In an instant, Google took me to the Immortality Institute, an organization dedicated to “conquering the blight of involuntary death.”

Not quite what I was looking for. So I hit the search again, but this time I took a look at the first ten results, etched in blue, green, and black against Google’s eternal white.

Nothing really caught my eye. Cryonics stuff, a business called Immortality Inc., pretty much what you might expect. I couldn’t put what I was looking for into words, but I knew this wasn’t it.

Then I noticed the advertising relegated to the right side of the screen. There were four ads, each no more than three lines of text. The first was someone who claimed to have met immortal ETs. Pass. The third and fourth were from eBay and Yahoo Shopping. These megasites had purchased the immortality keyword in some odd and obliquely interesting hope that people searching for immortality might well find relief through . . . buying shit online. (In fact, what Yahoo and eBay were doing was the equivalent of search arbitrage— buying top positions for a search term on Google and then creating a link to the exact same search term on their own sites, in the hope of capturing high-value customers).

Interesting, but I wasn’t looking to buy the concept of immortality; I wanted to understand it. I took a pass on those as well. But the second paid link pointed to the epic Gilgamesh, which I hazily recalled as the first story ever written down—in Sumerian cuneiform, if memory served. I clicked on the link, earning Google a few pennies in the process, and landed on an obscure bookseller’s page. The epic of Gilgamesh, the site instructed me, recounts mankind’s “longing stretch toward the infinite” and its “reluctant embrace of the temporal. This is the eternal lot of mankind.”

Bingo. I didn’t quite know why, but this was the stuff I was looking for. My vague desire to understand the concept of immortality had brought me to the epic of Gilgamesh, and now I was hooked. My search was bearing fruit. But I didn’t want to buy a book and wait for it to come. I was in the moment of discovery, the heat of possible consummation. I wanted to read that epic, right now.1 So I typed the title itself into Google, and once again found myself larded with options.

But this time the organic results (the search results in the middle of a Google page, as opposed to the ads on the right) nailed it: the first two offered direct translations of the stone tablets upon which the epic is written. Clicking on the first link, I found a Washington State University professor’s summary of the Gilgamesh story. It read:

Gilgamesh was an historical king of Uruk in Babylonia, on the River Euphrates in modern Iraq; he lived about 2700 b.c. Although historians . . . tend to emphasize Hammurabi and his code of law, the civilizations of the Tigris-Euphrates area, among the first civilizations, focus rather on  Gilgamesh and the legends accruing around him to explain, as it were, themselves. Many stories and myths were written about Gilgamesh, some of which were written down about 2000 b.c. in the Sumerian language on clay tablets which still survive . . . written in the script known as cuneiform, which means “wedge-shaped.” The fullest surviving version, from which the summary here is taken, is derived from twelve stone tablets . . . found in the ruins of the library of Ashurbanipal, king of Assyria, 669–633 b.c., at Nineveh. The library was destroyed by the Persians in 612 b.c., and all the tablets are damaged. The tablets actually name an author, which is extremely rare in the ancient world, for this particular version of the story: Shin-eqi-unninni. You are being introduced here to the oldest known human author we can name by name!

In my search for immortality, I had found the oldest known named author in the history of Western civilization. Thanks to the speed, vastness, and evanescent power of Google, I came to know his name and his work within thirty seconds of proffering a vaguely worded query. This man, Shin-eqi-unninni, now lived in my own mind. Through his writings, with an assist from Google and a university professor, he had, in a sense, become immortal.

But wait! There’smore. Gilgamesh’s story is one of man’s struggle with the concept of immortality, and the story itself was nearly lost in an act of literary vandalism—the destruction of a great king’s library. As I contemplated all of this, sensing that, just possibly, I had found a way to explain why search was so important to our culture.

I read the first tablet’s opening lines:

The one who saw all (Sha nagba imuru) I will declare to the world, The one who knew all I will tell about [line missing] He saw the great Mystery, he knew the Hidden: He recovered the knowledge of all the times before the Flood. He journeyed beyond the distant, he journeyed beyond exhaustion, And then carved his story on stone.

What does it mean, I wondered, to become immortal through words pressed in clay—or, as was the case here, through words formed in bits and transferred over the Web? Is that not what every person longs for—what Odysseus chose over Kalypso’s nameless immortality— to die, but to be known forever? And does not search offer the same immortal imprint: is not existing forever in the indexes of Google and others the modern-day equivalent of carving our stories into stone? For anyone who has ever written his own name into a search box and anxiously awaited the results, I believe the answer is yes.

Something to think about, anyway. Good luck, Mr. Levinson and Mr. Page. I’m cheering you on, even if I can’t quite explain why. Maybe it’s that missing line from Gilgamesh we’re all trying to find….

*Hat tip to one of my editors Bill Brazell, for pinging me as I was writing this about this very news.

  • Content Marquee

Hasta La Alta Vista, Baby.

By - June 29, 2013

I just saw the news that Yahoo! is “sunsetting” Alta Vista, one of the first “good” search engines. This makes me a little misty, as Alta Vista was the search engine I used BG – Before Google – and it had a real shot at *being* Google, had its various owners not utterly screwed it up over the years. Did you know, for example, that at one point Alta Vista was the largest and most widely used search tool on the web? Its driving force, Lois Monier, once told me “search should be a pencil” – he was adamant that Alta Vista not become a portal.

But Alta Vista was owned by DEC, a now dead computer company, which was bought by Compaq, another now dead computer company. And they made it a portal. And through the now defunct Overture, the assets of Alta Vista made their way to Yahoo!, a still alive portal. But now, Alta Vista is going to truly be dead.

It’s hard to watch an important player in the early Internet go away – and it makes me reflect on a couple of things. First, how much or our own culture and history we’re losing day by day, even despite the best efforts of archivists like Brewster Kahle. And secondly, on a personal note, Alta Vista was the search engine that helped me find my birth mother way back in 1995, when my wife was pregnant with our first child, and my life-long wonderings as an adoptee took on a new urgency. Alta Vista pointed me to an online forum for people like me, and there I found a person who helped me find my mother. Pretty cool.

So I’ll miss you, Alta Vista. And if any of you want to know the engine’s journey, well, there’s a book for that.

On Google Glass and OpenCo NYC

By - May 09, 2013

In case you have any interest, here’s a short clip of me opining on Google Glass and the upcoming OpenCoNYC, which is going to be HOT. More on that soon.

We’ve Seen This Movie Before…On Traffic of Good Intent

By - April 26, 2013

(image) Back in 2005 I whipped off a post with a title that has recently become relevant again – “Traffic of Good Intent.” That post keyed off  a major issue in the burgeoning search industry – click fraud. In the early days of search, click fraud was a huge problem (that link is from 2002!). Pundits (like me) claimed that because everyone was getting paid from fraud, it was “something of a whistling-past-the-graveyard issue for the entire (industry).” Cnet ran a story in 2004 identifying bad actors who created fake content, then ran robots over AdSense links on those pages. It blamed the open nature of the Web as fueling the fraudsters, and it noted that Google could not comment, because  it was in its quiet period before an IPO.

But once public, Google did respond, suing bad actors and posting extensive explanations of its anti-fraud practices. Conversely, a major fraud-based class action lawsuit was filed against all of the major search engines. Subsequent research suggested that as much as 30% of commercial clicks were fraudulent  – remember, this was after Google had gone public, and after the issue had been well-documented and endlessly discussed in the business and industry press. The major players in search finally banded together to fight the problem – understanding full well that without a united front and open communication, trust would never be established.

Think about that little history lesson – a massive, emerging new industry, one that was upending the entire marketing ecosystem, was operating under a constant cloud of “fraud” which may have been poisoning nearly a third of the revenues in the space. Yet billions in revenue and hundreds of billions in market value was still created. And after several years of lawsuits, negative press, and lord-knows-how-much-fraud, the clickfraud story has pretty much been forgotten.

Sound familiar?

It should. Because the same movie is once again playing, but this time the problem has migrated to the open ecosystem of programmatic display. As anyone who’s studied the LUMAscape knows, we now have a VC-fueled industry worth billions, with many players primed to go public in the coming year or so. And the original search players – Google in particular, but also Microsoft and Yahoo! – are also major actors in this new industry.

My post from January of this year – It’s Time To Call Out Fraud In The Adtech Ecosystem – summarized the new breed of fraud in our industry, and recently, many publications  have intensified their coverage of the topic. In late February, I invited a handful of adtech CEOs to a lunch where we discussed the issue, and everyone at the table – from AppNexus to Google, OpenX to MediaOcean – agreed that it was time to address the problem head on.

And that’s how we got to the news  this past week that the IAB is standing up a task force on “Traffic of Good Intent.” I’m proud to be a co-chair of the group (and yes, the name does come from that 2005 post in these pages). This time around, there are many more players, a much larger industry, and a far more complicated ecosystem. But it’s worth remembering that bad actors always take advantage of open systems. It’s up to us to unite and drive them back. We should all be trading in traffic of good intent – real human beings, engaged with real content and services across the Internet. Our customers, partners, investors, and our good company names depend on it.

I look forward to the work.

It’s Time To Call Out Fraud In The Adtech Ecosystem

By - January 26, 2013

A confusing landscape = ripe opportunities for fraud.

As part of research I’m doing both for the book and for my upcoming conference (the CM Summit, more on that soon), I’ve been in pretty extensive conversations lately with dozens of key players in the advertising technology industry. I find the ecosystem that has developed  to be fascinating, complex, and ripe with opportunity (and deeply important to the future of our society, not just marketing). I’ll be writing about it quite a bit in coming months. But before I do, I wanted to call out a growing issue that our industry will have to tackle sooner rather than later.

Just as in the early, wild west days of search (1999-2004), the programmatic advertising business – a multi-billion dollar marketplace growing faster than search, video, or anything else for that matter – is riddled with fraud.

That’s what many very reputable sources have told me in great length over the past few months. It’s something of an open secret, and more and more people are speaking out against it. Here’s Federated Media’s Walter Knapp on the problem, back in March of last year:

The great thing about the Internet is that it is built on the foundation of openness — from the way the domain system works to the way content and publishing are increasingly democratic. The core technologies embrace openness, sharing, linking and the ability to consume content across devices and across wired or wireless connections. Unfortunately, the openness we depend on in the digital media business is also available to people who can (and will) take advantage of this openness and exploit it for their own selfish wants.

Knapp notes two forms of fraud – ad injectors, fraudulent browser plugins that take over ad calls; and the practice of inserting an entire site into a 1×1 pixel hidden on high traffic but low quality sites featuring porn or music lyrics. Both are examples I’ve heard about over and over in my reporting. A third involves “stacking” ads one behind the other, all playing video to completion, often playing in inactive tabs. A fourth features refreshing ad calls on accelerated schedules or in inactive tabs. Yet another involves running as many ads as possible out of view, simply to gain “view through attribution” on a closed loop success metric.

More people are starting to call these practices out. AppNexus CEO Brian O’Kelly prominently featured the issue of fraud in his blog post celebrating his company’s recent $75 million funding, and what he intends to use it for:

Quality We will continue to invest in cleaning up the advertising marketplace. We’re proud of our anti-piracy stance, and our 5x volume growth this year indicates that you don’t need to serve on BitTorrent sites to be an ad platform company. We are investing heavily in fighting fraud, porn, malvertising, and malicious toolbars, and we are actively working on viewability tools.

Programmatic industry watcher AdExchanger puts it this way:

AppNexus’s pledge to invest money in ad quality issues is worth calling out. The issue is becoming more pervasive as companies emerge to exploit the vulnerabilities of real-time traded inventory to data and impression fraud, malvertising, and other nefarious practices. Fraudulent activities aside, the emergence of robust ad verification and viewability tools means display ad marketplaces and buying platforms must keep a clean nose.

It’s true that many folks are working on addressing the issue, including the IAB. But the bad actors are currently far ahead of the good guys, and worse, many in our industry are turning a blind eye, hoping the problem goes away in time, without too much publicity. Why? Well, nearly everyone gets paid from fraud – the publishers, the exchanges, the data providers, and the agencies. Even the marketers,who are footing the bill, feel like they are getting value – because the success metrics they’ve set up are being  met.

But fraud hurts the ecosystem in a massive way. It means that low quality, invisible, or purely fraudulent inventory is holding down the average value of the entire marketplace – hurting high quality, engaged publishers in the process, stunting investment in quality content.

Over and over, I hear that the reason CPMs (the amount of money a marketer is willing to pay for one thousand advertising impresssions) are so low is because “there’s infinite inventory.”

Hogwash. There’s only so much time in the day, and only so many pages where actual human beings are really paying attention, and the web (including mobile) is growing at a finite pace. There are even fewer places where marketers can be assured of quality, engagement, and appropriate context. It’s time we focus on identifying them, and ridding ourselves of the true source of “infinite inventory” – fraud.

Portrait of Twitter As A Young Media Company

By - January 21, 2013

Last year I predicted that Twitter would become a media company. However, I focused mainly on the new “Discover” functionality, and I probably should have gone a lot further. In this piece, I intend to.

So I’ll start with this: 2013 will be the year Twitter starts to create, curate, and co-create media experiences on top of its platform. I hinted at this in my brief coverage of Twitter’s Oscar Index (see Twitter’s Makin’ Media), but allow me to put a bit more flesh on the bones.

So what might one make from the fact that your platform captures hundreds of millions of individuals declaring what’s going on at any give time? Well, let’s break down some of the signals in all that supposed noise. As I’ve written over and over and over in the past several years, Twitter presents a massive search problem/opportunity. For example, Twitter’s gotten better and better at what’s called “entity extraction” – identifying a person, place, or thing, then associating behaviors and attributes around that thing. This (among other reasons) is why its Discover feature keeps getting better and better. Another important signal is location – Twitter is increasingly focused on getting us to geolocate our tweets. A third signal is the actual person tweeting – his or her influence and interest graph. Yet another signal is time – when was the entity tweeted about?

Real time entity extraction crossed with signals like those described above is the Holy Grail – and I’m guessing Twitter is almost, if not already there.

Once you get good at all these things (and more), a number of really interesting possibilities open up. Identifying “big things” that are going on at any given time is something that Twitter already does – though not particularly well (the best window in is the “Trends” box on the left of the page). Regardless, Twitter has become a go-to service for quick updates about news events (Sandy, Newtown, etc), entertainment events (SuperBowl, Oscars, Grammys, etc), and well….pretty much any kind of event.

But so far, it’s not exactly easy to get the big picture of what’s really going on for any given event on Twitter. In fact, it’s rather difficult. You can search for a hashtag, or keywords you think are associated with an event, but no matter what, it’s extremely difficult to makes sense of it all. For a big event like Sandy Hook or the Oscars, there are literally millions of tweets to sift through. And those tweets have millions of pictures, links, and videos. How can you know what’s important?

This is exactly the problem that  media experiences are designed to solve. By combining intelligent algorithms (these tweets are retweeted more than others, this video is linked to more than all the others, etc) and some smart editors, Twitter can (and most likely will) surface instant windows into events as they unfold around the globe. I imagine logging into Twitter at some point in the future and seeing a dashboard not of Trends, but of “Happenings” – Events edited to my interest graph, location, and the like. When I click on on of those events, I enter a meticulously edited media experience – a pulsing, ever changing feast of information tailored around that event.

So, put in one sentance: Twitter’s going to do events soon.

What other media experiences might Twitter create? Well, extending the logic, it only makes sense that Twitter will curate media services, just as LinkedIn and now Facebook are starting to do (I argue that Graph Search is a media play here).

“Just Landed” – from 2009.

As Google has proven, words have a lot of power on the web. They have even more power when put in context at scale. Consider what happened when a data artist asked a simple question: Where are people when then tweet that they “just landed”?

Now, imagine Twitter stands up a service that allows you to see patterns around phrases like “looking for someone to…,” or “just got a job,” or “python developer,” etc. Yep, lurking inside all that Twitter data is a pretty powerful job service. And I’m only using jobs as a straw man (and because it’s a driving force of LinkedIn’s success, of course). When you have humanity whispering into your ear at scale, you can tune in any number of valuable signals. Getting a job is one important signal. But so is getting married, buying a house or a car, graduating, and, and and….well you get the picture. Standing up “media services” around these life milestones is what media companies do. They used to be called magazines. What might Twitter call them? In 2013, we’ll most likely find out.

So far I’ve proposed two new media features of Twitter: Events and Media Services. I’ll round out this post with a prediction around a third: Video. Video is a vastly under-leveraged asset on Twitter, but people are sharing millions of links to video clips every day on the service. I imagine that Twitter will soon offer some kind of video curation feature – giving its base the ability to find the most popular videos based on pivot points of time, interest, and people. Surfacing and creating more video on the Twitter service has got to be a major priority at the company. And let’s not forget that Twitter bought Vine, after all…

After all, everybody loves video. In particular, advertisers love video. After all, Twitter is already working with Neilsen to become the official barometer of television conversations.

Which brings me to the “stick the landing” portion of this particular round up. Twitter is going to make much more media this year, because Twitter is going to make much more money this year. Each of the features I described above – Events, Media Services, and Video – bring with them inherent business models. I don’t expect they’ll look like traditional display models, of course, but I would not be surprised if they strayed a bit from Twitter’s current Promoted Suite products. With new media products come new advertising products. And new revenue.

Time will tell if I got this one right. Meanwhile, what do you think?

Facebook Is No Longer Flat: On Graph Search

By - January 15, 2013

A sample Graph Search result for the query “friends photos before 1999.”

By now the news is sweeping across the blogosophere and into the mainstream press: Facebook is doing Search!

Well, not so fast. Facebook is not doing search, at least not search Google-style. However, the world’s largest social network has radically re-engineered its native search experience, and the result is not only much, much better, it’s also changed my mind about the company’s long term future.

Yesterday, Tom Stocky, Facebook Product Management Director, and Lars Rasmussen, Engineering Director, gave me a sneak peek of today’s much anticipated announcement (it’s gonna be a phone! A new Newsfeed! A big acquisition!). So as to not bury the lead, Facebook has built what it’s calling “Graph Search,” a solidly conceived structured-search service which leverages the company’s massive trove of personal data in any number of new ways (some obvious, some nuanced, and some glaring omissions). But before I get to the details, I want to write about why this matters so much.

Prior to seeing the new search, I was not certain Facebook would ever live up to the hype it has accrued over its short life. It’s a good service, but it’s flat – over time, it struck me, people would tire of tending to it. They set up their social graph, toss a few sheep, poke some pals (or not), “like” this or that (often off-domain), waste hours on Farmville, and then…engagement drops slowly over time. I’m also not a fan of Facebook’s domain-specific approach to the world, as many of you know. Facebook’s new search doesn’t address Facebook’s walled garden mentality (yet), but it nails the first issue. Once this search product is rolled out to all of its members, Facebook will no longer be flat.

This is a big deal on many fronts. First and foremost, Facebook has an engagement problem, particularly in markets (like the US) where its use has become ubiquitous and many of its original users are two, three or more years into the “Facebook habit.” While the company doesn’t talk about this issue, I am confident it’s real (in private conversations with people at Facebook, it’s called the “set it up and forget it” problem). If people do not constantly feed Facebook with engagement, its value attenuates over time. As the service slows in overall growth, engagement with its current base becomes critical. New connections are the lifeblood of a service like Facebook. Without a steady stream of meaningful Likes, Friend Requests, declared Interests, and such, the platform would wither.

Put another way, Facebook needed a service that layered a fresh blanket of value over its core topography. Graph Search is it.

Zuckerberg’s Engagement

One sign of how important this new search is? According to the folks I spoke to yesterday, Facebook’s mercurial founder and CEO Mark Zuckerberg calls Search the “third pillar” of the company’s service, elevating it to the level of Newsfeed and Timeline, the two most important new features since Facebook’s launch (Open Graph is probably up there as well, but it’s true value remains locked up until there is mortar connecting it all, which Search could well be).

A team of engineers and product folk have been working on Graph Search for more than a year, and Zuckerberg has been engaged with them the entire time. The team has been in “lockdown” – a exclusive state of focus on one product so as to ship it as quickly as possible – for the past 34 days. Lockdown is a time honored and rather prestigious occurence inside Facebook, dating back to Zuckerberg’s original Facemash dorm room programming outburst. During the Search team lockdown, Stocky told me, he and Rasmussen got plenty of 2 AM emails and unexpected late night visits from the CEO.

In other words, this is A Really Big Deal for the company.

Why? Well, a quick tour of the product will explain.

What Is It? 

Graph Search subsumes Facebook’s previous search offering, which was extremely weak and focused mainly on the use case of navigation (finding people and pages).  The new service takes full advantage of the face that Facebook is, at its core, a massive structured database of tagged entities. The initial beta “indexes” four main types of these entities: People, Photos, Places, and Interests. Over time, I am told, Facebook will expand its index to include all Facebook posts and even the Open Graph – which means the “rest of the web.”

But for now, users can search across four main categories, using a slick set of intuitive verbs (“lives,” “like,” “work,” etc.), nouns (“San Francisco,” “Indian,” “restaurants,” “friends” etc.), prepositions (“before,” “with,” “in”) and pronouns (“who,” that,” etc.). This makes for a richly structured set of results: “Friends of friends who live in San Francisco and like Indian restaurants,” for example. Or “Friends who have been to Ireland,” or “Photos of friends before 1990.” Once you get the hang of it, the possible pivot points are endless, and the results are quite intriguing.

Stocky and Rasmussen, both ex-Googlers, walked me through a few intriguing use cases, one of which harkens back to one of Facebook’s original use cases – dating – and another which looks forward and presents a threat to LinkedIn’s current strength: Recruiting.

Let’s say you’re single, and you’re interested only in dating engineers who are also friends of your friends. With Graph Search, it’s ridiculously easy to find “friends of friends” who are also engineers. (And single, of course). You can look at their pictures, profiles, interests, and then ask for an introduction from whichever of your pals happens to be connected to one who looks like a good prospect (you could also just “poke” the guy if you wanted to…). Want only C++ programmers, or Indian C++ programmers, or  Indian C++ programmers under 35 years of age? Done.

Or, let’s say you work at, I dunno, Google. And you want to recruit product management talent from, say, Facebook. Again, the best way to get to that talent is probably a friend. So why not do a search for “friends of friends who work at Facebook and are product managers”? Why not, indeed.

One can imagine such functionality will create a lot of new engagement on the service. And not just from people “friending” prospective beaus or hires. Recall that when Google burst onto the scene, it prompted a dramatic response from owners of web pages, who immediately began rewiring their sites to be optimized for search. Similarly, Facebook’s Graph Search will incent Facebook users to “dress” themselves in better meta-data, so as to be properly represented in all those new structured results. People will start to update their profiles with more dates, photo tags, relationship statuses, and, and, and…you get the picture. No one wants to be left out of a consideration set, after all.

Facebook Gets More Weather

Last year I wrote a post titled “Facebook Is Now Making Its Own Weather.” The focus was on Facebook’s Newsfeed, and how an economy of value was now in place to game Facebook’s “edgerank” algorithm, which determines what stories show up in a person’s feed. With Graph Search, I expect a similar ecosystem will emerge. All of a sudden, two things will be true that previously were not: Facebook users will be using search, a lot, creating liquidity in Facebook “SERPS.” And secondly, there will be significant perceived value in being included in those search results, both for individuals (I want to be considered for that job at Google!) and for companies/brands (I want to message to anyone looking for a job!).

While Graph Search is in very early beta, I don’t think I’m going out on a limb by predicting that it won’t be long before Facebook integrates a product that lets marketers purchase ads in these new search results. It already has a similar product, which is by default included in suggested searches (the “auto completed” queries suggested to a user as they enter terms). At the moment, however, paid listings are not included in search results. They will be. Which means, of course, the rise of a native SEO/SEM ecosystem inside Facebook. Add in Open Graph search across the web, and presto…Google’s got some serious potential competition. (Well, not exactly presto. Incorporating Open Graph is going to take some serious chops and time. But still…).

Even without incorporation of Open Graph or Posts, Graph Search is going to change the game for brands and people on the Facebook service. As I watched Stocky and Rasmussen put their product through its paces, I couldn’t help but wonder how much new traffic the product will drive around the Facebook Platform. Will Facebook be watching “conversions” – clickthroughs from search results to profiles and pages? Of course they will! Will Facebook report those referrals to individuals and brands, much as Google Analytics does for webpage today? Not yet…but wait for it. It’ll come….

 What’s Missing: Sharing Results

I’ve already noted that Graph Search does not index content (posts) or the Open Graph, though I’m told that’s coming. But the big miss, from my point of view, is the inability to share search results.

Share search results? Who’d want to do that? Well, in web search, very few of us. That’s because with rare exception, open web search is not an inherently social action – it’s private and it’s ephemeral. But inside the walls of Facebook, it’s definitionally so. In fact, I’d argue that every single “result page” in Graph Search is a “media object” in its own right. If you search for “pictures of friends before 1990,” for example, you get the equivalent of a Pinterest board of your friends’ childhood shots. Wouldn’t you like to post that on your timeline so your pals can see it? Better yet, wouldn’t you like to export it to Pinterest or Tumblr? Of course you would (but, alas, I don’t expect Facebook will allow it, under cover of “protecting user privacy.” More on that in a second.)

Or take another example. Say you have a pal in Southern California who is despondent after being dumped by her boyfriend. You do a quick Graph Search for “single friends of friends under 30 who work in Los Angeles.” The results look pretty promising. Don’t you want to shoot them over to your pal with the subject line “Don’t despair, there are plenty of fish in the sea!” Of  course you do.

I mean, just a query like “Photos I Like” is a huge feature win for Facebook. And who wouldn’t want to post a montage of “Photos I Like” to their timeline? (Or, ahem, their personal blog?!)

For now, you can’t share the results of your searches with anyone else, and that’s a bug that should be a feature. When I brought up the issue, I was told that the privacy implications of sharing searches were extremely complicated. Because of past missteps and current scrutiny, Facebook is going to tread cautiously here (privacy was a central theme in Graph Search’s launch). I certainly understand why, but while those issues are sorted, I expect there are going to be a lot of screen shots of Graph search results being shared around the web.

Bigger privacy issues will likely arise around what might be called the Randi Zuckerberg principle – as in “Oh shit, I didn’t realize I’d show up in that circumstance!” Graph Search is going to expose all manner of privacy controls as super important, and send millions back to Facebook’s sometimes-confusing dashboards, so as to appropriately re-tool settings such that nothing untwoard shows up in this important new functionality.

And to me, this is a Very Good Thing. A couple of years ago, I wrote a post titled  The Rise of Digital Plumage in which I predicted that we’d all become habituated to “dressing” ourselves in structured data, so as to best present ourselves to the world at any given time. Graph Search is another important tool in our ever-growing digital wardrobe, one that motivates us to understand and manage the implications of our ever-expanding digital footprint.

Facebook just posted an announcement about its new search here.  The initial beta will roll out slowly, folks will have to ask to join a waitlist to get the service. I’ll be updating this post as the news is discussed and digested….

With Google’s 2012 Zeitgeist, You Won’t Learn Much. Why?

By - December 13, 2012

Guess what? This guy was big this year. Really!

I think readers know that on balance, I’m a fan of Google. I recently switched to the Nexus 4 (more coming on that front as I settle into really using it). I believe the company has a stronger core philosophy than many of its rivals. Overall, given that it’s nearly impossible to avoid putting your data into someone’s cloud, I believe that Google is probably the best choice for any number of reasons.

But that doesn’t mean I won’t criticize the company. And every year about this time, I end up doing just that.

Because the annual Google Zeitgeist came out this week, and I’ve spent a bit of time digging into it. And once again, I’m pretty disappointed.

In the past I’ve criticized Google for failing to ask interesting questions of the massive amount of data it collects on search patterns each year. Once again, this lament applies. I honestly do not care what top ten TV Shows, Sports Stars, Songs, or even People we collectively care about, because there is *never* a surprise in those results.

But Google knows so much more….and could really tease out some insights if it cared to. Imagine if Google took its massive search query database and worked with some of the leaders in the open data movement to mine true insights? Sure, Google would have to be careful about how it released the data, but the output would be extraordinary, I’d warrant.

Instead, we find out that Gangnam Style was a big deal this year. No shit!?

But it gets worse. Not only is Zeitgeist rife with pop culture fluff, as you drill down into it by country, eager perhaps to find something interesting, it turns out Google has chosen to eliminate certain potentially sensitive categories altogether.

For the US and most other countries, for example, there is a “What is….” category, which shows the top search queries that start with “What is…” For the US, the answers are

  1. What is SOPA
  2. What is Scientology
  3. What is KONY
  4. What is Yolo
  5. What is Instagram
  6. What is Pinterest
  7. What is Lent
  8. What is Obamacare
  9. What is iCloud
  10. What is Planking

But is there a “What Is…” for Saudi Arabia? Nope. China? Uh-uh. The United Arab Emirates? No sir. Egypt? Move along.

Hmmm.

Oddly, Google did provide “What is…” was for Singapore, where people living under that “benign dictatorship” were interested in the same things as the US –  “What is SOPA”,  “What is Scientology” and, for politicians, who is “Mitt Romney.”

For the US only, you can drill down into all manners of other categories past the main page, including News, Science, Tech, Humanities, and Cities. Those are pretty interesting categories, but Google only provides them for the US, which is a shame.

Furthermore, I find it interesting that Google, with all of its translation technology, does not have a translation button on the results pages for countries where the majority of the searches are in languages other than English. This is most likely due to political sensitivities, because if you run some of the results through Google Translate (do you believe I had to do that?!), you get some stuff that I am sure does not please the regimes of countries like China, Saudi Arabia, and the UAE.

For example, here are some of the top searches for Saudi Arabia, translated (roughly I am sure) by the Google Translate service:

Student outcomes Arab Idol insurance Ramadan Series 2012 Mohamed Morsi explosion Riyadh Burma Free Syrian Army Shura Council tornado Sandy

But again, you aren’t going to get much more insight into what Saudi folks are *really* thinking about, because Google failed to ask the interesting questions, like those it has in the “News” section of the US Zeitgeist. I’d sure be interested in “Political Gaffes,” “Election Issues,” and “News Sources,” in Saudi Arabia, China, or the UAE.

In fact, for Saudi Arabia, Google has ommitted the “Top News Searches” box that is on several of the other country pages (even Egypt). Instead, the topics for Saudi Arabia (besides trending searches and people) focus on sports and entertainment stars, fashion designers, TV shows, and the like. Deep, Google. Thanks.

Now, the datasets are different for each country, and it may be that Google simply didn’t have enough trending data to surface interesting political insights for these controversial countries.

Somehow, though, I don’t buy that. This set of lists feels extremely human vetted – I’m guessing an awful lot of hand wringing went into chosing what to show and what might prove problematic to Google’s best interests were it to see the light of day.

If that is the case, I urge the company to have more courage. I bet if Google open sourced its query data sets (eliminating any chance of PII getting out, of course), I bet academics, data scientists, and just plain interested folks would let loose an explosion of insight. Pop up the rainbird of data, Google, and let the ecosystem flourish. We’d all be the richer for it.

Facebook Is Now Making Its Own Weather

By - November 09, 2012

(image) The past month or so has seen the rise and fall of an interesting Internet tempest – the kind of story that gets widely picked up, then quickly amplified into storms of anger, then eventually dies down as the folks who care enough to dig into the facts figure out that the truth is somewhere outside the lines of the original headline-grabbing story.

The topic this time around centers on Facebook’s native ad unit called “Sponsored Stories,” and allegations that the company is gaming its “Edgerank” algorithm such that folks once accustomed to free promotion of their work on Facebook must now pay for that distribution.

Edgerank determines the posts you see in your Facebook newsfeed, and many sites noticed that sometime early this Fall, their traffic from Facebook shrank dramatically. Others claimed traffic had been declining since the Spring, but it wasn’t until this Fall that the story gained significant traction.

I’ve been watching all this play out – first via an angry post on the New York Observer site in which the author posits that Facebook is “broken on purpose” so as to harvest Sponsored Story revenue. An even angrier post on the same theme came five weeks later on a site called Dangerous Minds. From it:

Spring of 2012 was when bloggers, non-profits, indie bands, George Takei, community theaters, photographers, caterers, artists, mega-churches, high schools, tee-shirt vendors, campus coffee shops, art galleries, museums, charities, food trucks, and a near infinite variety of organizations; individuals from all walks of life; and businesses, both large and small, began to detect—for it was almost imperceptible at first—that the volume was getting turned down on their Facebook reach. Each post was now being seen only by a fraction of their total “fans” who would previously have seen them.

The author goes on to argue that Facebook was breaking the implicit contract between himself – an independent blogger – and Facebook, the corporation.

…as a publisher of a medium readership blog, I used to get a great deal from using Facebook—but I understood it to be a two-way reciprocal arrangement because I was driving traffic back to Facebook as well, and reinforcing their brand awareness with prominent widgets on our blog.

Now, if you’ve read my Thneeds post, you know I’m sympathetic to this point of view. I believe large social platforms like Facebook and Twitter “harvest” content from the Indpendent Web, and leverage the traffic and engagement that this content creates on their platforms to their own benefit via scaled advertising offerings. Most of us are fine with the deal – we promote our work on social sites, social sites drive traffic back to us. We like that traffic, either just because we like more folks reading our work, or, in the case of commercial sites like this one, because we serve ads against it.

Now, as I’ve noted many times over the past six months, this bargain is breaking down, because it’s getting harder and harder to monetize traffic using standard display advertising units. That’s not Facebook’s problem, per se, it’s ours. (See here for my suggestions as to how to solve it).

Nevertheless, for many sites, the spectre of losing significant traffic from Facebook means a serious blow to revenues. And from the point of view of the Dangerous Minds blogger, Facebook first cut his traffic off, then began asking him to pay to get it back (in the form of promoting his posts via Sponsored Stories).

This makes for a very good narrative: corporate greed laid bare. It got picked up by a lot of sites, including Ars Technica and even the aforementioned George Takei, who is upset that he’s lost the ability to push his posts to all 2.9 million of his Facebook fans.

Turns out, the truth is a lot more complicated. I’ve done some reporting on this issue, but not nearly as much as TechCrunch did. In a follow up to the Dangerous Minds story, TechCrunch claimed to have debunked the entire story. Titled Killing Rumors With Facts: No, Facebook Didn’t Decrease Page Feed Reach To Sell More Promoted Posts, the story argues that Facebook didn’t change its algorithms to drive up revenue, but rather to cull “spammy posts” from folks’ newsfeeds.

Facebook has always shown just a percentage of all possible posts in a given person’s newsfeed. Anyone paying attention already knew that. The company uses its Edgerank algorithm to determine what it thinks might be interesting to an individual, and sometime in the past few months, I can confirm through sources which wish to remain anonymous that Facebook made a pretty significant change to Edgerank that penalized posts that it felt were not high quality.

Of course, that begs the question: How does Facebook determine what “quality” is? The answer, in the main, is by measuring engagement – is the post shared, liked, clicked on, etc? If so, then it is seen as quality. If not, it’s demoted in value.

Is this sounding familiar to anyone yet? In short, Facebook just executed a Panda.

I held back from writing anything till this predictable cycle played out, because I had a theory, one that I believe is now confirmed: Facebook is now making its own weather, just like Google, and in the past couple months, we’ve witnessed the first widespread instance of a Facebook weather event.

For those of you who don’t know quite what I’m talking about, a bit of history. Ten or so years ago, the ecosystem around search began to notice shifts in how Google drove traffic around the web. Google would make a change to its algorithms, and all of a sudden some sites would see their traffic plummet (other sites sometimes saw the opposite occur). It seemed to those injured that the only way to get their Google traffic back was to buy Google AdWords – corporate greed laid bare. This story played out over and over, to the point where the weather events started to get names, just like hurricanes do. (The first was called Boston).

Early last year Google made a major change to its algorithms that penalized what it believed was lower quality content. Dubbed “Panda,” the changes targeted “content farms” that cranked out SEO friendly pages as AdWords bait. This had dramatic effects on many sites that specialized in “gaming” Google. It also hit sites that weren’t necessarily playing that game – updates like Panda often create collateral damage. Over time, and as it always does, Google fine-tuned Panda until the ecosystem stabilized.

I believe that Facebook is now learning how to manage its own weather. I don’t know the Dangerous Minds website well enough to know if it deserved the drop in traffic that occurred when Facebook had its Panda moment. But one thing does strike me as interesting to note: A significant drop in traffic means a particular site is losing audience that has proactively decided to click on a link inside their newsfeed. That click means the person leaves Facebook and goes to the the Dangerous Minds site. To me, that’s a pretty serious sign of engagement.

However, one might argue that such a signal is not as important to Facebook as internal ones such as “liking” or “sharing” across the Facebook network. To that end, I am sure we’ve not heard the last round of serious grumbling that Facebook is gaming its own Edgerank algorithm to benefit Facebook’s internal goals – to the detriment of the “rest of the web.” Be they publishers or folks like George Takei, who after all wants to push his Facebook fans to any  number of external links where they might buy his books or sign up to meet him at the next Comic Con, the rest of the web depends on “social traffic” from Facebook. The question is, should they optimize for that traffic, or will their efforts be nullified in the next Edgerank update?

Facebook is learning how to tread the delicate line between its own best interests, and those of its users – and the Internet That Is Not Facebook. Google does this every day – but it has a long history as a distributor of traffic off its main site. Facebook, not so much. Over time, the company will have to decide what kind of a relationship it wants to have with the “rest of the web.” It will probably have to start engaging more openly with its own ecosystem, providing guidance on best practices and how to avoid being penalized. This is a practice that took Google years to hone, and many still think the company has a lot of work to do.

Regardless, Facebook is now making its own weather. Now comes the fun part: Trying to predict it.

Bing Tries Harder, But For Me, It’s A Draw

By - September 06, 2012

It’s not easy being number two. As a marketer, you have limited choices – you can pretend you’re not defined by the market leader, or, you can embrace your position and go directly after your nemesis.

For years, Bing executives have privately complained about how hard it is to “break the Google habit,” even as they refused to market directly against Google. They were Avis, always trying harder.

No more. Today Microsoft announced its “Bing It On” challenge, a direct descendant of the iconic Pepsi challenge more than 30 years ago (the fact that I still remember that marketing campaign, and feel good about it, is a testament to its power).

It’s always a risk to ask consumers to test products blind, side by side, but Bing is doing it: Right here at “Bingiton.com.”

I bit and took the challenge – how did it go?

My first query has been my baseline for more than ten years – my own name (“john battelle“). Yeah, it’s a vanity search, but all of us have very strong opinions about what comes up when we put our names into search.

The winner? It was close, but Bing won. Its results seemed fresher – the Google screen had stuff about me from eBay and the BusinessWeek exchange in the first page (I never use eBay, and haven’t been active on that BusinessWeek page for more than two years). The Bing side also had my LinkedIn profile, which I consider important, though it also had an old picture of me flipping off the camera from 1998 (that’s getting very old), and a picture of a former business partner who isn’t me at all.

My second search – the misspelled (on purpose) “bset hotels sydney” made me question how the results were being delivered to the test site. Given how much I know about Google’s SERPs, it was pretty easy for me to tell which side was Google (it’s the left – the giveaway is the list of hotels with integrated reviews). But the results didn’t look quite like I was used to at Google. Here’s a comparison:

The main reason? This test had stripped out Google’s Maps feature for some reason, which certainly penalized the page from a visual and utility standpoint. Doesn’t seem like a fair fight.

So I gave that one a draw and moved onto another search.

Next up I tried a search I know both engines have had a bit of trouble with. I often lose the URL of my son’s boy scout troop, and have to search around for it a bit – it used to be buried in a nested Web 1.0 service, but recently was updated with its own URL, which unfortunately has terrible SEO. My first query usually doesn’t work, but it leads me in the right direction. It’s been a year or so since I’ve tried this (my son is older now), so I thought this might be a fresh search with some history to it. The query is “troop 43 larkspur california“.

The winner was most certainly Google. It found the old website (which has been impossible to find in the past) and the new one built in the last two years.

My next query was very utilitarian. My dad had a scare last night and is staying overnight at the hospital. I need to call the main line to check how he’s doing. So I entered “marin general hospital phone.” I figure if you put the word “phone” in there, the search engine should understand I need the phone number.

The Bing results had the number in the snippet of the first result. Google had it broken out clearly, but as the fourth result. Again, I know on Google I always get a map. But there was no map in these results. Also, I know that Bing prides itself on breaking out phone numbers, but I didn’t see the familiar Bing phone breakout box. Oh well, I had to go with Bing, because the information I needed was surfaced in the first result.

So going into my last search, it was two for Bing, one draw, and one for Google.

 

My last test was “winter rentals stinson beach” – a search I’ve done recently – and with some frustration – as I am taking a place there to write over the winter. I know what good results look like here, given I’ve done a lot of poking around already. It was relatively easy for me to pick a winner. It was Google, which filtered out most of the single home entries (I don’t want to find one home, I want to find listings with lots of them) and it also highlighted services and a local realtor I happen to know has the best inventory in the area.

So for me, the test concluded as a draw – two wins for Bing, two for Google, and one disqualification. Not exactly the two-to-one ratio in favor of Bing that Microsoft claims is the average, but then again, not bad either.

Remember, this is an entirely non scientific and subjective “test.” And of course, this test by its nature must exclude any personalization, search history, or other important bells and whistles that search engines use to tailor results to ongoing clients.

In the end, Bing proved to me that it deserves to be considered equal to Google for a variety of use cases. I don’t know if that’s enough to break the Google habit, but it certainly will get folks talking. And that’s an important part of marketing, isn’t it?!