free html hit counter Future of Search Archives | Page 2 of 8 | John Battelle's Search Blog

It’s Time To Call Out Fraud In The Adtech Ecosystem

By - January 26, 2013

A confusing landscape = ripe opportunities for fraud.

As part of research I’m doing both for the book and for my upcoming conference (the CM Summit, more on that soon), I’ve been in pretty extensive conversations lately with dozens of key players in the advertising technology industry. I find the ecosystem that has developed  to be fascinating, complex, and ripe with opportunity (and deeply important to the future of our society, not just marketing). I’ll be writing about it quite a bit in coming months. But before I do, I wanted to call out a growing issue that our industry will have to tackle sooner rather than later.

Just as in the early, wild west days of search (1999-2004), the programmatic advertising business – a multi-billion dollar marketplace growing faster than search, video, or anything else for that matter – is riddled with fraud.

That’s what many very reputable sources have told me in great length over the past few months. It’s something of an open secret, and more and more people are speaking out against it. Here’s Federated Media’s Walter Knapp on the problem, back in March of last year:

The great thing about the Internet is that it is built on the foundation of openness — from the way the domain system works to the way content and publishing are increasingly democratic. The core technologies embrace openness, sharing, linking and the ability to consume content across devices and across wired or wireless connections. Unfortunately, the openness we depend on in the digital media business is also available to people who can (and will) take advantage of this openness and exploit it for their own selfish wants.

Knapp notes two forms of fraud – ad injectors, fraudulent browser plugins that take over ad calls; and the practice of inserting an entire site into a 1×1 pixel hidden on high traffic but low quality sites featuring porn or music lyrics. Both are examples I’ve heard about over and over in my reporting. A third involves “stacking” ads one behind the other, all playing video to completion, often playing in inactive tabs. A fourth features refreshing ad calls on accelerated schedules or in inactive tabs. Yet another involves running as many ads as possible out of view, simply to gain “view through attribution” on a closed loop success metric.

More people are starting to call these practices out. AppNexus CEO Brian O’Kelly prominently featured the issue of fraud in his blog post celebrating his company’s recent $75 million funding, and what he intends to use it for:

Quality We will continue to invest in cleaning up the advertising marketplace. We’re proud of our anti-piracy stance, and our 5x volume growth this year indicates that you don’t need to serve on BitTorrent sites to be an ad platform company. We are investing heavily in fighting fraud, porn, malvertising, and malicious toolbars, and we are actively working on viewability tools.

Programmatic industry watcher AdExchanger puts it this way:

AppNexus’s pledge to invest money in ad quality issues is worth calling out. The issue is becoming more pervasive as companies emerge to exploit the vulnerabilities of real-time traded inventory to data and impression fraud, malvertising, and other nefarious practices. Fraudulent activities aside, the emergence of robust ad verification and viewability tools means display ad marketplaces and buying platforms must keep a clean nose.

It’s true that many folks are working on addressing the issue, including the IAB. But the bad actors are currently far ahead of the good guys, and worse, many in our industry are turning a blind eye, hoping the problem goes away in time, without too much publicity. Why? Well, nearly everyone gets paid from fraud – the publishers, the exchanges, the data providers, and the agencies. Even the marketers,who are footing the bill, feel like they are getting value – because the success metrics they’ve set up are being  met.

But fraud hurts the ecosystem in a massive way. It means that low quality, invisible, or purely fraudulent inventory is holding down the average value of the entire marketplace – hurting high quality, engaged publishers in the process, stunting investment in quality content.

Over and over, I hear that the reason CPMs (the amount of money a marketer is willing to pay for one thousand advertising impresssions) are so low is because “there’s infinite inventory.”

Hogwash. There’s only so much time in the day, and only so many pages where actual human beings are really paying attention, and the web (including mobile) is growing at a finite pace. There are even fewer places where marketers can be assured of quality, engagement, and appropriate context. It’s time we focus on identifying them, and ridding ourselves of the true source of “infinite inventory” – fraud.

  • Content Marquee

Portrait of Twitter As A Young Media Company

By - January 21, 2013

Last year I predicted that Twitter would become a media company. However, I focused mainly on the new “Discover” functionality, and I probably should have gone a lot further. In this piece, I intend to.

So I’ll start with this: 2013 will be the year Twitter starts to create, curate, and co-create media experiences on top of its platform. I hinted at this in my brief coverage of Twitter’s Oscar Index (see Twitter’s Makin’ Media), but allow me to put a bit more flesh on the bones.

So what might one make from the fact that your platform captures hundreds of millions of individuals declaring what’s going on at any give time? Well, let’s break down some of the signals in all that supposed noise. As I’ve written over and over and over in the past several years, Twitter presents a massive search problem/opportunity. For example, Twitter’s gotten better and better at what’s called “entity extraction” – identifying a person, place, or thing, then associating behaviors and attributes around that thing. This (among other reasons) is why its Discover feature keeps getting better and better. Another important signal is location – Twitter is increasingly focused on getting us to geolocate our tweets. A third signal is the actual person tweeting – his or her influence and interest graph. Yet another signal is time – when was the entity tweeted about?

Real time entity extraction crossed with signals like those described above is the Holy Grail – and I’m guessing Twitter is almost, if not already there.

Once you get good at all these things (and more), a number of really interesting possibilities open up. Identifying “big things” that are going on at any given time is something that Twitter already does – though not particularly well (the best window in is the “Trends” box on the left of the page). Regardless, Twitter has become a go-to service for quick updates about news events (Sandy, Newtown, etc), entertainment events (SuperBowl, Oscars, Grammys, etc), and well….pretty much any kind of event.

But so far, it’s not exactly easy to get the big picture of what’s really going on for any given event on Twitter. In fact, it’s rather difficult. You can search for a hashtag, or keywords you think are associated with an event, but no matter what, it’s extremely difficult to makes sense of it all. For a big event like Sandy Hook or the Oscars, there are literally millions of tweets to sift through. And those tweets have millions of pictures, links, and videos. How can you know what’s important?

This is exactly the problem that  media experiences are designed to solve. By combining intelligent algorithms (these tweets are retweeted more than others, this video is linked to more than all the others, etc) and some smart editors, Twitter can (and most likely will) surface instant windows into events as they unfold around the globe. I imagine logging into Twitter at some point in the future and seeing a dashboard not of Trends, but of “Happenings” – Events edited to my interest graph, location, and the like. When I click on on of those events, I enter a meticulously edited media experience – a pulsing, ever changing feast of information tailored around that event.

So, put in one sentance: Twitter’s going to do events soon.

What other media experiences might Twitter create? Well, extending the logic, it only makes sense that Twitter will curate media services, just as LinkedIn and now Facebook are starting to do (I argue that Graph Search is a media play here).

“Just Landed” – from 2009.

As Google has proven, words have a lot of power on the web. They have even more power when put in context at scale. Consider what happened when a data artist asked a simple question: Where are people when then tweet that they “just landed”?

Now, imagine Twitter stands up a service that allows you to see patterns around phrases like “looking for someone to…,” or “just got a job,” or “python developer,” etc. Yep, lurking inside all that Twitter data is a pretty powerful job service. And I’m only using jobs as a straw man (and because it’s a driving force of LinkedIn’s success, of course). When you have humanity whispering into your ear at scale, you can tune in any number of valuable signals. Getting a job is one important signal. But so is getting married, buying a house or a car, graduating, and, and and….well you get the picture. Standing up “media services” around these life milestones is what media companies do. They used to be called magazines. What might Twitter call them? In 2013, we’ll most likely find out.

So far I’ve proposed two new media features of Twitter: Events and Media Services. I’ll round out this post with a prediction around a third: Video. Video is a vastly under-leveraged asset on Twitter, but people are sharing millions of links to video clips every day on the service. I imagine that Twitter will soon offer some kind of video curation feature – giving its base the ability to find the most popular videos based on pivot points of time, interest, and people. Surfacing and creating more video on the Twitter service has got to be a major priority at the company. And let’s not forget that Twitter bought Vine, after all…

After all, everybody loves video. In particular, advertisers love video. After all, Twitter is already working with Neilsen to become the official barometer of television conversations.

Which brings me to the “stick the landing” portion of this particular round up. Twitter is going to make much more media this year, because Twitter is going to make much more money this year. Each of the features I described above – Events, Media Services, and Video – bring with them inherent business models. I don’t expect they’ll look like traditional display models, of course, but I would not be surprised if they strayed a bit from Twitter’s current Promoted Suite products. With new media products come new advertising products. And new revenue.

Time will tell if I got this one right. Meanwhile, what do you think?

Facebook Is No Longer Flat: On Graph Search

By - January 15, 2013

A sample Graph Search result for the query “friends photos before 1999.”

By now the news is sweeping across the blogosophere and into the mainstream press: Facebook is doing Search!

Well, not so fast. Facebook is not doing search, at least not search Google-style. However, the world’s largest social network has radically re-engineered its native search experience, and the result is not only much, much better, it’s also changed my mind about the company’s long term future.

Yesterday, Tom Stocky, Facebook Product Management Director, and Lars Rasmussen, Engineering Director, gave me a sneak peek of today’s much anticipated announcement (it’s gonna be a phone! A new Newsfeed! A big acquisition!). So as to not bury the lead, Facebook has built what it’s calling “Graph Search,” a solidly conceived structured-search service which leverages the company’s massive trove of personal data in any number of new ways (some obvious, some nuanced, and some glaring omissions). But before I get to the details, I want to write about why this matters so much.

Prior to seeing the new search, I was not certain Facebook would ever live up to the hype it has accrued over its short life. It’s a good service, but it’s flat – over time, it struck me, people would tire of tending to it. They set up their social graph, toss a few sheep, poke some pals (or not), “like” this or that (often off-domain), waste hours on Farmville, and then…engagement drops slowly over time. I’m also not a fan of Facebook’s domain-specific approach to the world, as many of you know. Facebook’s new search doesn’t address Facebook’s walled garden mentality (yet), but it nails the first issue. Once this search product is rolled out to all of its members, Facebook will no longer be flat.

This is a big deal on many fronts. First and foremost, Facebook has an engagement problem, particularly in markets (like the US) where its use has become ubiquitous and many of its original users are two, three or more years into the “Facebook habit.” While the company doesn’t talk about this issue, I am confident it’s real (in private conversations with people at Facebook, it’s called the “set it up and forget it” problem). If people do not constantly feed Facebook with engagement, its value attenuates over time. As the service slows in overall growth, engagement with its current base becomes critical. New connections are the lifeblood of a service like Facebook. Without a steady stream of meaningful Likes, Friend Requests, declared Interests, and such, the platform would wither.

Put another way, Facebook needed a service that layered a fresh blanket of value over its core topography. Graph Search is it.

Zuckerberg’s Engagement

One sign of how important this new search is? According to the folks I spoke to yesterday, Facebook’s mercurial founder and CEO Mark Zuckerberg calls Search the “third pillar” of the company’s service, elevating it to the level of Newsfeed and Timeline, the two most important new features since Facebook’s launch (Open Graph is probably up there as well, but it’s true value remains locked up until there is mortar connecting it all, which Search could well be).

A team of engineers and product folk have been working on Graph Search for more than a year, and Zuckerberg has been engaged with them the entire time. The team has been in “lockdown” – a exclusive state of focus on one product so as to ship it as quickly as possible – for the past 34 days. Lockdown is a time honored and rather prestigious occurence inside Facebook, dating back to Zuckerberg’s original Facemash dorm room programming outburst. During the Search team lockdown, Stocky told me, he and Rasmussen got plenty of 2 AM emails and unexpected late night visits from the CEO.

In other words, this is A Really Big Deal for the company.

Why? Well, a quick tour of the product will explain.

What Is It? 

Graph Search subsumes Facebook’s previous search offering, which was extremely weak and focused mainly on the use case of navigation (finding people and pages).  The new service takes full advantage of the face that Facebook is, at its core, a massive structured database of tagged entities. The initial beta “indexes” four main types of these entities: People, Photos, Places, and Interests. Over time, I am told, Facebook will expand its index to include all Facebook posts and even the Open Graph – which means the “rest of the web.”

But for now, users can search across four main categories, using a slick set of intuitive verbs (“lives,” “like,” “work,” etc.), nouns (“San Francisco,” “Indian,” “restaurants,” “friends” etc.), prepositions (“before,” “with,” “in”) and pronouns (“who,” that,” etc.). This makes for a richly structured set of results: “Friends of friends who live in San Francisco and like Indian restaurants,” for example. Or “Friends who have been to Ireland,” or “Photos of friends before 1990.” Once you get the hang of it, the possible pivot points are endless, and the results are quite intriguing.

Stocky and Rasmussen, both ex-Googlers, walked me through a few intriguing use cases, one of which harkens back to one of Facebook’s original use cases – dating – and another which looks forward and presents a threat to LinkedIn’s current strength: Recruiting.

Let’s say you’re single, and you’re interested only in dating engineers who are also friends of your friends. With Graph Search, it’s ridiculously easy to find “friends of friends” who are also engineers. (And single, of course). You can look at their pictures, profiles, interests, and then ask for an introduction from whichever of your pals happens to be connected to one who looks like a good prospect (you could also just “poke” the guy if you wanted to…). Want only C++ programmers, or Indian C++ programmers, or  Indian C++ programmers under 35 years of age? Done.

Or, let’s say you work at, I dunno, Google. And you want to recruit product management talent from, say, Facebook. Again, the best way to get to that talent is probably a friend. So why not do a search for “friends of friends who work at Facebook and are product managers”? Why not, indeed.

One can imagine such functionality will create a lot of new engagement on the service. And not just from people “friending” prospective beaus or hires. Recall that when Google burst onto the scene, it prompted a dramatic response from owners of web pages, who immediately began rewiring their sites to be optimized for search. Similarly, Facebook’s Graph Search will incent Facebook users to “dress” themselves in better meta-data, so as to be properly represented in all those new structured results. People will start to update their profiles with more dates, photo tags, relationship statuses, and, and, and…you get the picture. No one wants to be left out of a consideration set, after all.

Facebook Gets More Weather

Last year I wrote a post titled “Facebook Is Now Making Its Own Weather.” The focus was on Facebook’s Newsfeed, and how an economy of value was now in place to game Facebook’s “edgerank” algorithm, which determines what stories show up in a person’s feed. With Graph Search, I expect a similar ecosystem will emerge. All of a sudden, two things will be true that previously were not: Facebook users will be using search, a lot, creating liquidity in Facebook “SERPS.” And secondly, there will be significant perceived value in being included in those search results, both for individuals (I want to be considered for that job at Google!) and for companies/brands (I want to message to anyone looking for a job!).

While Graph Search is in very early beta, I don’t think I’m going out on a limb by predicting that it won’t be long before Facebook integrates a product that lets marketers purchase ads in these new search results. It already has a similar product, which is by default included in suggested searches (the “auto completed” queries suggested to a user as they enter terms). At the moment, however, paid listings are not included in search results. They will be. Which means, of course, the rise of a native SEO/SEM ecosystem inside Facebook. Add in Open Graph search across the web, and presto…Google’s got some serious potential competition. (Well, not exactly presto. Incorporating Open Graph is going to take some serious chops and time. But still…).

Even without incorporation of Open Graph or Posts, Graph Search is going to change the game for brands and people on the Facebook service. As I watched Stocky and Rasmussen put their product through its paces, I couldn’t help but wonder how much new traffic the product will drive around the Facebook Platform. Will Facebook be watching “conversions” – clickthroughs from search results to profiles and pages? Of course they will! Will Facebook report those referrals to individuals and brands, much as Google Analytics does for webpage today? Not yet…but wait for it. It’ll come….

 What’s Missing: Sharing Results

I’ve already noted that Graph Search does not index content (posts) or the Open Graph, though I’m told that’s coming. But the big miss, from my point of view, is the inability to share search results.

Share search results? Who’d want to do that? Well, in web search, very few of us. That’s because with rare exception, open web search is not an inherently social action – it’s private and it’s ephemeral. But inside the walls of Facebook, it’s definitionally so. In fact, I’d argue that every single “result page” in Graph Search is a “media object” in its own right. If you search for “pictures of friends before 1990,” for example, you get the equivalent of a Pinterest board of your friends’ childhood shots. Wouldn’t you like to post that on your timeline so your pals can see it? Better yet, wouldn’t you like to export it to Pinterest or Tumblr? Of course you would (but, alas, I don’t expect Facebook will allow it, under cover of “protecting user privacy.” More on that in a second.)

Or take another example. Say you have a pal in Southern California who is despondent after being dumped by her boyfriend. You do a quick Graph Search for “single friends of friends under 30 who work in Los Angeles.” The results look pretty promising. Don’t you want to shoot them over to your pal with the subject line “Don’t despair, there are plenty of fish in the sea!” Of  course you do.

I mean, just a query like “Photos I Like” is a huge feature win for Facebook. And who wouldn’t want to post a montage of “Photos I Like” to their timeline? (Or, ahem, their personal blog?!)

For now, you can’t share the results of your searches with anyone else, and that’s a bug that should be a feature. When I brought up the issue, I was told that the privacy implications of sharing searches were extremely complicated. Because of past missteps and current scrutiny, Facebook is going to tread cautiously here (privacy was a central theme in Graph Search’s launch). I certainly understand why, but while those issues are sorted, I expect there are going to be a lot of screen shots of Graph search results being shared around the web.

Bigger privacy issues will likely arise around what might be called the Randi Zuckerberg principle – as in “Oh shit, I didn’t realize I’d show up in that circumstance!” Graph Search is going to expose all manner of privacy controls as super important, and send millions back to Facebook’s sometimes-confusing dashboards, so as to appropriately re-tool settings such that nothing untwoard shows up in this important new functionality.

And to me, this is a Very Good Thing. A couple of years ago, I wrote a post titled  The Rise of Digital Plumage in which I predicted that we’d all become habituated to “dressing” ourselves in structured data, so as to best present ourselves to the world at any given time. Graph Search is another important tool in our ever-growing digital wardrobe, one that motivates us to understand and manage the implications of our ever-expanding digital footprint.

Facebook just posted an announcement about its new search here.  The initial beta will roll out slowly, folks will have to ask to join a waitlist to get the service. I’ll be updating this post as the news is discussed and digested….

With Google’s 2012 Zeitgeist, You Won’t Learn Much. Why?

By - December 13, 2012

Guess what? This guy was big this year. Really!

I think readers know that on balance, I’m a fan of Google. I recently switched to the Nexus 4 (more coming on that front as I settle into really using it). I believe the company has a stronger core philosophy than many of its rivals. Overall, given that it’s nearly impossible to avoid putting your data into someone’s cloud, I believe that Google is probably the best choice for any number of reasons.

But that doesn’t mean I won’t criticize the company. And every year about this time, I end up doing just that.

Because the annual Google Zeitgeist came out this week, and I’ve spent a bit of time digging into it. And once again, I’m pretty disappointed.

In the past I’ve criticized Google for failing to ask interesting questions of the massive amount of data it collects on search patterns each year. Once again, this lament applies. I honestly do not care what top ten TV Shows, Sports Stars, Songs, or even People we collectively care about, because there is *never* a surprise in those results.

But Google knows so much more….and could really tease out some insights if it cared to. Imagine if Google took its massive search query database and worked with some of the leaders in the open data movement to mine true insights? Sure, Google would have to be careful about how it released the data, but the output would be extraordinary, I’d warrant.

Instead, we find out that Gangnam Style was a big deal this year. No shit!?

But it gets worse. Not only is Zeitgeist rife with pop culture fluff, as you drill down into it by country, eager perhaps to find something interesting, it turns out Google has chosen to eliminate certain potentially sensitive categories altogether.

For the US and most other countries, for example, there is a “What is….” category, which shows the top search queries that start with “What is…” For the US, the answers are

  1. What is SOPA
  2. What is Scientology
  3. What is KONY
  4. What is Yolo
  5. What is Instagram
  6. What is Pinterest
  7. What is Lent
  8. What is Obamacare
  9. What is iCloud
  10. What is Planking

But is there a “What Is…” for Saudi Arabia? Nope. China? Uh-uh. The United Arab Emirates? No sir. Egypt? Move along.

Hmmm.

Oddly, Google did provide “What is…” was for Singapore, where people living under that “benign dictatorship” were interested in the same things as the US –  “What is SOPA”,  “What is Scientology” and, for politicians, who is “Mitt Romney.”

For the US only, you can drill down into all manners of other categories past the main page, including News, Science, Tech, Humanities, and Cities. Those are pretty interesting categories, but Google only provides them for the US, which is a shame.

Furthermore, I find it interesting that Google, with all of its translation technology, does not have a translation button on the results pages for countries where the majority of the searches are in languages other than English. This is most likely due to political sensitivities, because if you run some of the results through Google Translate (do you believe I had to do that?!), you get some stuff that I am sure does not please the regimes of countries like China, Saudi Arabia, and the UAE.

For example, here are some of the top searches for Saudi Arabia, translated (roughly I am sure) by the Google Translate service:

Student outcomes Arab Idol insurance Ramadan Series 2012 Mohamed Morsi explosion Riyadh Burma Free Syrian Army Shura Council tornado Sandy

But again, you aren’t going to get much more insight into what Saudi folks are *really* thinking about, because Google failed to ask the interesting questions, like those it has in the “News” section of the US Zeitgeist. I’d sure be interested in “Political Gaffes,” “Election Issues,” and “News Sources,” in Saudi Arabia, China, or the UAE.

In fact, for Saudi Arabia, Google has ommitted the “Top News Searches” box that is on several of the other country pages (even Egypt). Instead, the topics for Saudi Arabia (besides trending searches and people) focus on sports and entertainment stars, fashion designers, TV shows, and the like. Deep, Google. Thanks.

Now, the datasets are different for each country, and it may be that Google simply didn’t have enough trending data to surface interesting political insights for these controversial countries.

Somehow, though, I don’t buy that. This set of lists feels extremely human vetted – I’m guessing an awful lot of hand wringing went into chosing what to show and what might prove problematic to Google’s best interests were it to see the light of day.

If that is the case, I urge the company to have more courage. I bet if Google open sourced its query data sets (eliminating any chance of PII getting out, of course), I bet academics, data scientists, and just plain interested folks would let loose an explosion of insight. Pop up the rainbird of data, Google, and let the ecosystem flourish. We’d all be the richer for it.

Facebook Is Now Making Its Own Weather

By - November 09, 2012

(image) The past month or so has seen the rise and fall of an interesting Internet tempest – the kind of story that gets widely picked up, then quickly amplified into storms of anger, then eventually dies down as the folks who care enough to dig into the facts figure out that the truth is somewhere outside the lines of the original headline-grabbing story.

The topic this time around centers on Facebook’s native ad unit called “Sponsored Stories,” and allegations that the company is gaming its “Edgerank” algorithm such that folks once accustomed to free promotion of their work on Facebook must now pay for that distribution.

Edgerank determines the posts you see in your Facebook newsfeed, and many sites noticed that sometime early this Fall, their traffic from Facebook shrank dramatically. Others claimed traffic had been declining since the Spring, but it wasn’t until this Fall that the story gained significant traction.

I’ve been watching all this play out – first via an angry post on the New York Observer site in which the author posits that Facebook is “broken on purpose” so as to harvest Sponsored Story revenue. An even angrier post on the same theme came five weeks later on a site called Dangerous Minds. From it:

Spring of 2012 was when bloggers, non-profits, indie bands, George Takei, community theaters, photographers, caterers, artists, mega-churches, high schools, tee-shirt vendors, campus coffee shops, art galleries, museums, charities, food trucks, and a near infinite variety of organizations; individuals from all walks of life; and businesses, both large and small, began to detect—for it was almost imperceptible at first—that the volume was getting turned down on their Facebook reach. Each post was now being seen only by a fraction of their total “fans” who would previously have seen them.

The author goes on to argue that Facebook was breaking the implicit contract between himself – an independent blogger – and Facebook, the corporation.

…as a publisher of a medium readership blog, I used to get a great deal from using Facebook—but I understood it to be a two-way reciprocal arrangement because I was driving traffic back to Facebook as well, and reinforcing their brand awareness with prominent widgets on our blog.

Now, if you’ve read my Thneeds post, you know I’m sympathetic to this point of view. I believe large social platforms like Facebook and Twitter “harvest” content from the Indpendent Web, and leverage the traffic and engagement that this content creates on their platforms to their own benefit via scaled advertising offerings. Most of us are fine with the deal – we promote our work on social sites, social sites drive traffic back to us. We like that traffic, either just because we like more folks reading our work, or, in the case of commercial sites like this one, because we serve ads against it.

Now, as I’ve noted many times over the past six months, this bargain is breaking down, because it’s getting harder and harder to monetize traffic using standard display advertising units. That’s not Facebook’s problem, per se, it’s ours. (See here for my suggestions as to how to solve it).

Nevertheless, for many sites, the spectre of losing significant traffic from Facebook means a serious blow to revenues. And from the point of view of the Dangerous Minds blogger, Facebook first cut his traffic off, then began asking him to pay to get it back (in the form of promoting his posts via Sponsored Stories).

This makes for a very good narrative: corporate greed laid bare. It got picked up by a lot of sites, including Ars Technica and even the aforementioned George Takei, who is upset that he’s lost the ability to push his posts to all 2.9 million of his Facebook fans.

Turns out, the truth is a lot more complicated. I’ve done some reporting on this issue, but not nearly as much as TechCrunch did. In a follow up to the Dangerous Minds story, TechCrunch claimed to have debunked the entire story. Titled Killing Rumors With Facts: No, Facebook Didn’t Decrease Page Feed Reach To Sell More Promoted Posts, the story argues that Facebook didn’t change its algorithms to drive up revenue, but rather to cull “spammy posts” from folks’ newsfeeds.

Facebook has always shown just a percentage of all possible posts in a given person’s newsfeed. Anyone paying attention already knew that. The company uses its Edgerank algorithm to determine what it thinks might be interesting to an individual, and sometime in the past few months, I can confirm through sources which wish to remain anonymous that Facebook made a pretty significant change to Edgerank that penalized posts that it felt were not high quality.

Of course, that begs the question: How does Facebook determine what “quality” is? The answer, in the main, is by measuring engagement – is the post shared, liked, clicked on, etc? If so, then it is seen as quality. If not, it’s demoted in value.

Is this sounding familiar to anyone yet? In short, Facebook just executed a Panda.

I held back from writing anything till this predictable cycle played out, because I had a theory, one that I believe is now confirmed: Facebook is now making its own weather, just like Google, and in the past couple months, we’ve witnessed the first widespread instance of a Facebook weather event.

For those of you who don’t know quite what I’m talking about, a bit of history. Ten or so years ago, the ecosystem around search began to notice shifts in how Google drove traffic around the web. Google would make a change to its algorithms, and all of a sudden some sites would see their traffic plummet (other sites sometimes saw the opposite occur). It seemed to those injured that the only way to get their Google traffic back was to buy Google AdWords – corporate greed laid bare. This story played out over and over, to the point where the weather events started to get names, just like hurricanes do. (The first was called Boston).

Early last year Google made a major change to its algorithms that penalized what it believed was lower quality content. Dubbed “Panda,” the changes targeted “content farms” that cranked out SEO friendly pages as AdWords bait. This had dramatic effects on many sites that specialized in “gaming” Google. It also hit sites that weren’t necessarily playing that game – updates like Panda often create collateral damage. Over time, and as it always does, Google fine-tuned Panda until the ecosystem stabilized.

I believe that Facebook is now learning how to manage its own weather. I don’t know the Dangerous Minds website well enough to know if it deserved the drop in traffic that occurred when Facebook had its Panda moment. But one thing does strike me as interesting to note: A significant drop in traffic means a particular site is losing audience that has proactively decided to click on a link inside their newsfeed. That click means the person leaves Facebook and goes to the the Dangerous Minds site. To me, that’s a pretty serious sign of engagement.

However, one might argue that such a signal is not as important to Facebook as internal ones such as “liking” or “sharing” across the Facebook network. To that end, I am sure we’ve not heard the last round of serious grumbling that Facebook is gaming its own Edgerank algorithm to benefit Facebook’s internal goals – to the detriment of the “rest of the web.” Be they publishers or folks like George Takei, who after all wants to push his Facebook fans to any  number of external links where they might buy his books or sign up to meet him at the next Comic Con, the rest of the web depends on “social traffic” from Facebook. The question is, should they optimize for that traffic, or will their efforts be nullified in the next Edgerank update?

Facebook is learning how to tread the delicate line between its own best interests, and those of its users – and the Internet That Is Not Facebook. Google does this every day – but it has a long history as a distributor of traffic off its main site. Facebook, not so much. Over time, the company will have to decide what kind of a relationship it wants to have with the “rest of the web.” It will probably have to start engaging more openly with its own ecosystem, providing guidance on best practices and how to avoid being penalized. This is a practice that took Google years to hone, and many still think the company has a lot of work to do.

Regardless, Facebook is now making its own weather. Now comes the fun part: Trying to predict it.

Bing Tries Harder, But For Me, It’s A Draw

By - September 06, 2012

It’s not easy being number two. As a marketer, you have limited choices – you can pretend you’re not defined by the market leader, or, you can embrace your position and go directly after your nemesis.

For years, Bing executives have privately complained about how hard it is to “break the Google habit,” even as they refused to market directly against Google. They were Avis, always trying harder.

No more. Today Microsoft announced its “Bing It On” challenge, a direct descendant of the iconic Pepsi challenge more than 30 years ago (the fact that I still remember that marketing campaign, and feel good about it, is a testament to its power).

It’s always a risk to ask consumers to test products blind, side by side, but Bing is doing it: Right here at “Bingiton.com.”

I bit and took the challenge – how did it go?

My first query has been my baseline for more than ten years – my own name (“john battelle“). Yeah, it’s a vanity search, but all of us have very strong opinions about what comes up when we put our names into search.

The winner? It was close, but Bing won. Its results seemed fresher – the Google screen had stuff about me from eBay and the BusinessWeek exchange in the first page (I never use eBay, and haven’t been active on that BusinessWeek page for more than two years). The Bing side also had my LinkedIn profile, which I consider important, though it also had an old picture of me flipping off the camera from 1998 (that’s getting very old), and a picture of a former business partner who isn’t me at all.

My second search – the misspelled (on purpose) “bset hotels sydney” made me question how the results were being delivered to the test site. Given how much I know about Google’s SERPs, it was pretty easy for me to tell which side was Google (it’s the left – the giveaway is the list of hotels with integrated reviews). But the results didn’t look quite like I was used to at Google. Here’s a comparison:

The main reason? This test had stripped out Google’s Maps feature for some reason, which certainly penalized the page from a visual and utility standpoint. Doesn’t seem like a fair fight.

So I gave that one a draw and moved onto another search.

Next up I tried a search I know both engines have had a bit of trouble with. I often lose the URL of my son’s boy scout troop, and have to search around for it a bit – it used to be buried in a nested Web 1.0 service, but recently was updated with its own URL, which unfortunately has terrible SEO. My first query usually doesn’t work, but it leads me in the right direction. It’s been a year or so since I’ve tried this (my son is older now), so I thought this might be a fresh search with some history to it. The query is “troop 43 larkspur california“.

The winner was most certainly Google. It found the old website (which has been impossible to find in the past) and the new one built in the last two years.

My next query was very utilitarian. My dad had a scare last night and is staying overnight at the hospital. I need to call the main line to check how he’s doing. So I entered “marin general hospital phone.” I figure if you put the word “phone” in there, the search engine should understand I need the phone number.

The Bing results had the number in the snippet of the first result. Google had it broken out clearly, but as the fourth result. Again, I know on Google I always get a map. But there was no map in these results. Also, I know that Bing prides itself on breaking out phone numbers, but I didn’t see the familiar Bing phone breakout box. Oh well, I had to go with Bing, because the information I needed was surfaced in the first result.

So going into my last search, it was two for Bing, one draw, and one for Google.

 

My last test was “winter rentals stinson beach” – a search I’ve done recently – and with some frustration – as I am taking a place there to write over the winter. I know what good results look like here, given I’ve done a lot of poking around already. It was relatively easy for me to pick a winner. It was Google, which filtered out most of the single home entries (I don’t want to find one home, I want to find listings with lots of them) and it also highlighted services and a local realtor I happen to know has the best inventory in the area.

So for me, the test concluded as a draw – two wins for Bing, two for Google, and one disqualification. Not exactly the two-to-one ratio in favor of Bing that Microsoft claims is the average, but then again, not bad either.

Remember, this is an entirely non scientific and subjective “test.” And of course, this test by its nature must exclude any personalization, search history, or other important bells and whistles that search engines use to tailor results to ongoing clients.

In the end, Bing proved to me that it deserves to be considered equal to Google for a variety of use cases. I don’t know if that’s enough to break the Google habit, but it certainly will get folks talking. And that’s an important part of marketing, isn’t it?!

What Is Search Now? Disjoined.

By - August 31, 2012

(image shutterstock)

Today I answered a question in email for a reporter who works for Wired UK. He asked smart questions, as I would expect from a Wired writer. (Some day I’ll tell you all my personal story of Wired UK – I lived over there for the better part of a year back in 1997, trying to make that magazine work. I mostly failed – but it’s up and running strong now.)

In any case, one question in particular struck me. The writer is preparing a piece on the future of search. (I’ll link to it when it comes out). What big problems, he asked, still plague search?

That got me thinking. Here’s my answer:

The largest issue with search is that we learned about it when the web was young, when the universe was “complete” – the entire web was searchable! Now our digital lives are utterly fractured – in apps, in walled gardens like Facebook, across clunky interfaces like those in automobiles or Comcast cable boxes. Re-uniting our digital lives into one platform that is “searchable” is to me the largest problem we face today. 

It may be worth expanding on that sentiment. When it broke out in the mid 1990s, the web was society’s first at-scale digital artifact.  It spread in orders of ten, first thousands, then millions, then hundreds of millions of pages – and on it went, to the billions it now encompasses. Everybody wanted to “be” on the web – a creator class started making pages and companies and services, a consumer class started “surfing” this vast new digital object, and our collective conscience marvelled at what we had created together: millions of small pieces loosely joined. And the key and unappreciated point is this: those pieces were indeed joined.

It was that joining – through links, of course – that made search possible, that created what is unquestionably the most powerful and lasting new company of the past 20 years – Google.* But as I wrote in Why Hath Google Forsaken Us? A Meditation, Google’s core model – built on the open, linked world of the web – is under threat from the advance of the iPhone and the app, the Facebook and the Path, the automobile console, the Xbox, the cable box, and countless other “unlinked” digital artifacts.

Google knows this. Why else invest so much in Android, in Google+, in Motorola (it’s not just phones, it’s also cable boxes), in self-driving cars, for goodness sake? Google wants a foothold wherever digital information is created and shared, and man, are we creating a sh*t ton of it. Problem is, we’re not making it easy – or even possible – to link all this stuff together, should we care to.

Which takes me back to that core question the Wired reporter asked me: What’s the biggest problem plaguing search? In short, it’s that our digital world is no longer small pieces loosely joined. It’s also big chunks separate and apart. And that makes search – in its most broadest interpretation – damn near impossible.

Which leads to another question: What then, is search? Of course, the Wired reporter asked me that as well. My answer:

Search is now more than a web destination and a few words plugged into a box. Search is a mode, a method of interaction with the physical and virtual worlds. What is Siri but search? What are apps like Yelp or Foursquare, but structured search machines? Search has become embedded into everything, and has reached well beyond its web-based roots.

So we all search now, all the time, across all manner of artifacts, large and small. But our searches are not federated – we can’t search across these repositories, as we could across that wonderful, vast, loosely joined early world of the web. We’ve lost the connection.

Call me a fool, but I think the need for that connection will be so strong, that in time, we’ll sew all our digital artifacts back together again. At least, I certainly hope we will. Right now, it ain’t looking so likely – what with patent wars, wagon circling by big platforms, and the like. But I’m an optimist – and I hope you are as well.

* Sorry but Facebook isn’t there – yet. And Microsoft and Apple, well, they may make a play for that crown either 20 years ago or 20 years hence, but if you ask me for the most important company ever that launched as a native web business, the answer is indisputably Google.

 

 

 

Here We Go Again: The Gray Market in Twitter and Facebook

By - August 07, 2012

So, casually reading through this Fast Company story about sexy female Twitter bots, I come across this astounding, unsubstantiated claim:

My goal was to draw a straight line from a Twitter bot to the real, live person whose face the bot had stolen. In the daily bot wars–the one Twitter fights every day, causing constant fluctuations in follower counts even as brands’ followers remain up to 48% bot–these women are the most visible and yet least acknowledged victims…

There it was, tossed in casually, almost as if it was a simple cost of doing business – nearly half of the followers of major brands could well be “bots.”

The article focuses on finding a pretty woman whose image had been hijacked, sure, but what I found most interesting (but sadly unsurprising) was how it pointed to a site that promises to a thousand  followers to anyone who pays…wait for it…about $17. Yes, the site is real. And no, you shouldn’t be surprised, in the least, that such services exist.

It has always been so.

Back when I was reporting for The Search, I explored the gray market that had sprung up around Google (and still flourishes, despite Google’s disputed attempts to beat it back). Fact is, wherever there is money to be made, and ignorance or desperation exists in some measure, shysters will flourish. And a further fact is this: Marketers, faced with CMO-level directives to “increase my follower/friend counts,” will turn to the gray market. Just as they did back in the early 2000s, when the directive was “make me rank higher in search.”

Earlier this week I got an email from a fellow who has been using Facebook to market his products. He was utterly convinced that nearly all the clicks he’s received on his ad were fake – bots, he thought, that were programmed to make his campaigns look as if they were performing well. He was further convinced that Facebook was running a scam – running bot networks to drive performance metrics. I reminded him that Facebook was a public company run by people I believed were well intentioned, intelligent people who knew that such behavior, if discovered, would ruin both their reputation as well as that of the company.

Instead, I suggested, he might look to third parties he might be working with – or, hell, he might just be the victim of a drive-by shooting – poorly coded bots that just click on ad campaigns, regardless of whose they might be.

In short, I very much doubt Facebook (or Twitter) are actively driving fraudulent behavior on their networks. In fact, they have legions of folks devoted to foiling such efforts.Yet there is absolutely no doubt that an entire, vibrant ecosystem is very much engaged in gaming these services. And just like Google had at the dawn of search marketing, Twitter and Facebook have a very – er – complicated relationship with these fraudsters. On the one hand, the gray hats are undermining the true value of these social networks. But on the other, well, they seem to help important customers hit their Key Performance Indicators, driving very real money into company coffers, either directly or indirectly.

I distinctly recall a conversation with a top Google official in 2005, who – off the record – defended AdSense-splattered domain-squatters as “providing a service to folks who typed the wrong thing into the address bar.” Uh huh.

As long as marketers are obsessed with hollow metrics like follower counts, Likes, and unengaged “plays,” this ecosystem will thrive.

What truly matters, of course, is engagement that can be measured beyond the actions of bots. It is coming. But not before millions of dollars are siphoned off by the opportunists who have always lived on the Internet’s gray edge.

On Mayer, Yahoo!, and The (Other) Customer

By - July 18, 2012

Mayer at the Web 2 Summit, San Francisco

(image James Duncan Davidson)

I try to let big news percolate for a few days before weighing in, and it seems even more appropriate to follow that playbook when it came to the scrum around Marissa Mayer joining Yahoo.

Yes, I’ve known both Marissa Mayer (and Ross Levinsohn) professionally, for more than a decade, but so do many other folks, and it seems nearly all of them – Steven Levy and Kara Swisher intelligently among them – have weighed in, multiple times, on what this all means. If you want a rundown, just search for “Marissa Mayer” in Google News.

The coverage has taken its usual course from “Holy Shit!” to “What Will Happen to Ross?” to “Wait, Is Mayer Right for the Job” to “Here’s Our Advice/The Things That Need  to Be Fixed/What Mayer’s Focus Should Be” types of pieces.

This won’t really be any of those. Instead, I find myself thinking about the things I’ve not really seen much coverage of, at least in depth. And true to what I’ve spent a fair amount of time thinking about, they all come down to the intersection of media and technology, and the role marketing plays in that landscape.

When I spoke to Mayer after she was named CEO, I asked the question, almost as a joke – “So is Yahoo! a media or a technology company?” She was quick to respond that she just does not get the debate – of course it’s both. What matters, she pressed, is creating great products that surprise and delight Yahoo! customers.

I couldn’t agree more, yet there is an important nuance here – just who *are* Yahoo’s customers?

Let me step back here and posit something that might upset more than a few of you: Yahoo has two sets of customers, and of course the “end user” is one of them. But the other is the marketer.  And media companies – or “tech companies driven by media revenues,” or however else one might want to phrase it – sometimes ignore this fact at their peril.

I’ll let those of you who find such a statement anathema go ahead and click away – here’s a nice unicorn chaser if you’d like – or you can flame me in the comments (I do respond to most, as long as they’re in English and don’t employ more than the occasional insult).

But those of you who’ve continued to read probably know that I believe, deeply, that commercial publishing is a conversation between three key parties: The reader (or viewer), the publisher/content creator, and the marketer. And while it’s generally been true that this conversation has been all kinds of broken during much of the web’s history, the truth is, it needn’t be that way. Six years ago (!) I wrote a series of posts describing the rise of conversational media and imploring that marketers learn to join the conversation. I think it’s fair to say that this is happening, at scale.

Beyond the contributions of pioneers like Federated Media (yes, I had to plug us), the rise of “native” advertising formats is proof of this. Twitter’s promoted suite is one growing example, as is Facebook’s Sponsored Stories (and its attendant focus on getting brands to be true publishers on the Facebook platform). Pinterest, WordPress (in partnership with FM), and Tumblr are hard at work on “native” solutions for their services as well. All of these advertising solutions pale, however, in comparison to the original “native” advertising format of the Web: Google AdWords.

Many have pointed out that Mayer’s principle weakness, when compared to Levinsohn, is her lack of traditional media and marketing chops. I can say from very deep experience that the marketing business is very much a relationship business – CMOs and agency leaders live in a world driven by ideas, creative and content – and they want to know the people who they do business with, and trust them in a way that is difficult to model algorithmically. Mayer’s detractors point out that she’s not spent much time wooing Madison Avenue, or dealing with the inevitable headaches born of the complex, people-driven businesses that are agencies, marketing clients, and content partners.

While there is some truth in this criticism, I think it overlooks a few things. First and foremost, Mayer is a very fast study, and she already knows how important the traditional media business is to Yahoo. Hell, a quick overview of the company’s financials bears this out, as does a visit to any of its properties, which are dominated by advertising. Yahoo may have a lot of technology behind the covers, but its products are nearly all media products – content intended to gather an audience and provide a place for marketers to message to that audience. More than half of Yahoo’s revenues come from “display” advertising, most of the rest comes from search, which is also marketer driven.

Secondly, Mayer will be a big draw of talent, and not just engineering talent. She understands that if she can’t retain Levinsohn and/or his recent CRO Michael Barrett (I certainly hope she can), she’ll need to attract top tier media minds to the business. And I think she’ll succeed at doing just that.

But to me, the thing many are missing is that Mayer will bring her fanatical product focus to more than just Yahoo’s consumer-facing media offerings. She’ll also be staring at the company’s advertising products, and asking this simple question: How can we do better?

To answer that question, Mayer will need to do more than study the data (though of course, that will be important). She’ll need to sit down with a wide swath of Yahoo’s marketing customers and ask them what they want from their investment in her platform. She’ll hear an awful lot of conflicting advice, but it’s in the bricollage from all the feedback that the best ideas come out. Mayer can’t afford to immediately tack away from all those boxes and rectangles cluttering up the Yahoo! experience, nor should she – it turns out that display advertising does indeed work for marketers. But the larger question remains: Can we do better?

The answer lies in executing the subtle and ongoing iterative work of true digital publisher – improving the core product experience both sets of customers – consumers of the media experience, as well as marketers looking to be part of that experience in a more native fashion. And again, from a quick study of Yahoo’s products, there’s plenty of improvements to be made.

An important and related part of the work ahead for Mayer and her team will be deciding what role ad tech and search will play in Yahoo’s future. Despite purchasing Right Media back in 2007, Yahoo has never been seen as a leader in ad tech, and word on the street in the weeks prior to Mayer’s ascension was that Yahoo was about to outsource its ad technology platform to market leader Google. Of course, such a move is fraught with regulatory and business implications. And Mayer may well decide it’s in Yahoo’s best interest to invest in own its own destiny when it comes to the machine-driven world of ad serving and programmatic audience buying. But trust me, what Yahoo does here will be an extremely important directional indicator.

Which brings us to search. It’s been widely reported that Yahoo’s 2009 deal to outsource core search to Microsoft hasn’t worked out as well as either party wished it would. Given how important search is to Yahoo overall, and how deeply knowledgeable Mayer is in this particular field, I’d expect big changes in Yahoo Search. The company recently unveiled a new search product called “Axis,” which seems like a neat idea but feels a bit too complicated for most consumers to really grok. Mayer will likely take Occam’s Razor to search, and I expect the results will be quite positive.

But it’s the other side of Yahoo’s revenue equation – the branded display market – where Mayer will face her greatest challenges, and find her biggest opportunities. Yahoo isn’t a startup like Pinterest, Tumblr,  or even Twitter, where founders can leverage massive user growth to raise enough capital to “figure out how best to implement appropriate native marketing solutions.” Yahoo is nearly 20 years old, and it’s got a very deep, tangled, and somewhat tarnished brand in the minds of its best advertising customers. It’s true that creating world-beating consumer-facing products will go a long way toward fixing that brand. But those products must be informed by – and even created for – both sets of customers – the consumers of content, as well as those who pay for them to be created in the first place.

Larry Page Makes His Case

By - April 05, 2012

Given the headlines, questions, and legal actions Google has faced recently, many folks, including myself, have been wondering when Google’s CEO Larry Page would take a more public stance in outlining his vision for the company.

Well, today marks a shift of sorts, with the publication of a lenthy blog post from Larry titled, quite uninterestingly, 2012 Update from the CEO.

I’ve spent the past two days at Amazon and Microsoft, two Google competitors (and partners), and am just wrapping up a last meeting. I hope to read Page’s post closely and give you some analysis as soon as I can. Meanwhile, a few top line thoughts and points:

- Page pushes Google+ as a success, citing more than 100 million users, but still doesn’t address the question of whether the service is truly being used organically, rather than as a byproduct of interactions with other Google products. I’m not sure it matters, but it’s a question many have raised. He also doesn’t address, directly, the tempest over the integration of G+ into search.

- Page also does not directly address the issue of FTC privacy investigations into the company, not surprising, given any company’s response to these investigations is usually “no comment.” However, Google might have explained with a bit more gusto the reasons for its recent changes.

- Page tosses out another big number, this one around Android: 850K activations a day. Take that, Apple!

- Page uses the words “love” and “beauty” – which I find both refreshing and odd.

- Page also talks about making big bets, focusing on fewer products, and how it’s OK to not be exactly sure how big bets are going to make money. This is a topic where Google has a ton of experience, to be sure.

More when I get out of my last meeting….