free html hit counter Future of Search Archives | John Battelle's Search Blog

Google’s Year End Zeitgeist: Once Again, Insights Lacking

By - December 17, 2013
Zeitgeist13

Great photo, but not one we searched for….

It’s become something of a ritual – every year Google publishes its year-end summary of what the world wants, and every year I complain about how shallow it is, given what Google *really* knows about what the world is up to.

At least this year Google did a good job of turning its data into a pretty media experience. There are endless scrolling visual charts, there’s a emotional, highly produced video, and there’s a ton of lists to explore once you drill down. But there’s also a Google+ integration that frankly, was utterly confusing. Called #my2013 Gallery (sorry, there’s no link for it), it showed photos from a bunch of people I didn’t know, then invited me to add my own. Not sure what that was about. The “Search Trends Globe” shows top search terms by location, but you can’t click through to see results. Odd.

So kudos to Google for giving us a lot of eye candy – there are top ten lists for all manner of categories, from dog breeds to NFL teams to memes – all by geography. But the search capacity is, well – confusing. Once you search inside what you think is the year end Zeitgeist, you end up getting Google Trends data, and you’re kind of lost, not sure if you’re in the year-end special anymore. Bummer.

And while there are far more lists than I’ve seen before, there’s still no … insight. Even the “What is…” function, which was an interesting, if limited feature from last year’s Zeitgeist, is gone this year, most likely a victim of political correctness. (For why, see my post about last year’s Zeitgeist).

I sure wish Google would surprise us with Zeitgeist, but once again, no dice.

  • Content Marquee

Traffic of Good Intent: We Beat Fraud By Working Together

By - December 06, 2013

TOGIscreen

Earlier this year I wrote a post titled It’s Time To Call Out Fraud In The Adtech Ecosystem. The overwhelming response to that riposte led to a lunch at this year’s IAB annual meeting, which then led to the formation of the Traffic of Good Intent task force (TOGI), an IAB-sanctioned working group composed of leaders from nearly every major player in the media and adtech industry. We’ve made a lot of progress since our first informal luncheon meeting nine months ago – I think the issue of fraud is now a top priority in our industry, and we continue work on best practices, solutions, and education. Today marks a milestone for our industry, the release of two white papers. Both are clearly written and intended to catalyze our progress to date.

Understanding Online Traffic Fraud gives a broad overview of the problem, laying out definitions of non-human traffic, and lays out half a dozen reasons you should give a sh*t. For me, the money quote is this: “Failing to root out traffic fraud funds criminal activity and supports organized crime.” Because as an advocate for publishers, that’s what fraud is: it’s stealing. It’s taking money and value out of the pockets of publishers, and putting it into the pockets of criminals. Along the way, any number of intermediaries also make money, and in the short term, they may be incented to continue to do so. We have to change that.

The second document, Traffic Fraud: Best Practices for Reducing Risks to Exposure, details actions all player in this ecosystem – brands, agencies, trading desks, technology providers, exchanges, publishers and more – can do to clean up this pervasive problem.

If you buy, sell, or traffic in online advertising, please read these documents, and help us move the needle even further. Fraud is not a problem that can be solved by pointing fingers or blaming one side or the other. We have to work together – and these documents are living proof that we can.

Apple+Topsy: It’s Not About Twitter (And Twitter Is Probably Cool With That)

By - December 03, 2013

TopsyApple

I’m going out on a limb, but a fairly stout one: Like Azeem, I think Apple bought Topsy for its search chops. But Azeem, who I admire greatly, says Topsy could become the search engine “for iOS… to index both the social Web, but also the best bits of the Web that power Siri and Apple Maps, [and] reduce the reliance on Google and reduce the flow of advertising dollars to the big G.” Certainly possible, but I don’t think Apple bought Topsy for its ability to search the web, or even for its trove of Twitter data. That might be a nice bonus, but I don’t think it’s the bogey.* Others have written that Topsy might be used to improve Apple’s iTunes/app search, but again, I think that’s not thinking big enough.

No, Apple most likely bought Topsy because Topsy has the infrastructure to address one of Apple’s biggest problems: the iOS interface. Let’s face it, iOS (and the app-based interface in general) is slowly becoming awful. It’s like the web before good search showed up.  To move to the next level, Apple needs a way to improve how its customers interact with iOS. Topsy will help them get there. Also, I think Twitter is happy that Apple bought Topsy – but more on that later.

Let me explain. First, my statement that iOS is “becoming awful.” Faithful readers know I’m not a fan of iOS. I switched to Android almost two years ago, and I’ve never looked back. But it’s not as if the Android interface is much better – I just like its chances of developing into something more powerful down the line. In the past few years, I’ve written several posts about the kind of interface I believe needs to emerge across mobile (which until last year, Apple pretty much dominated). Given my  obsession with the topic, it’s probably no secret that I view mobile’s biggest problem boils down to one of search.

In  Apple Won’t Build a (Web) Search Engine and Of Course Apple Is Going to Do Search, I argued that Apple must get into the “app search” game. Just as web search became the coin of the web realm, app search will be next. It won’t look like web search, I argued, but at its core, it’s quite similar.

That was three years ago, right after Apple bought Siri, launched iAds, and was relentlessly touting the growth of its app ecosystem. I was certain Apple was going to figure out a way to create value above the level of a particular app, using all that tasty data it had within its restrictive walled garden to build the next generation iOS interface.

But so far, Apple has failed to innovate inside its own ecosystem (unless you count minimalist icons and bright base colors as innovation). Three years later, we’re still stuck in a user interface of app-filled screens, most of which we never use, each disconnected  from the other save for the fact they happen to reside on your phone, possibly right next to each other, but otherwise unaware of the value they might reap should they magically start sharing links and data with each other. (You know, the way the web works.)

This has to change.

Google knows it, which is why I find Google Now so fascinating. Apple knows it too – the days of home screens littered with app icons are numbered. What will replace it?

My guess is some kind of intelligent, search-driven interface that “understands” you, based on the intent you signal through your use of all kinds of apps – including browser apps, of course, as well as true search apps like Siri (or Google Now). This new kind of interface responds to your voice as well as your location, your history, and anything else you might willingly (or unwittingly) feed it. It will strive to always put the very thing you need at your fingertips – something that simply isn’t possible without understanding your interactions as the equivalent of …. well, a personal interest graph.

And to do that, Apple needs a powerful engine, the kind of engine that, say, has been hard at work understanding a massive corpus of interest data for, say, six or so years. Something like Topsy.

My prediction: Apple doesn’t really care about Twitter data. The more I think about it, the more I’d wager that Twitter most likely blessed Apple’s purchase – and why not. With its newfound post-IPO billions, Twitter could have easily forced Topsy’s price well past $200 million. But Twitter is probably thrilled that Apple bought Topsy – Apple just took out a company that Twitter eventually would have had to either buy or kill. Now, Twitter is free to build enterprise value on top of its own data, as well it should, and Apple has a team of engineers who I’m guessing can’t wait to get their hands on a new kind of tweet stream – all that structured data captured, but not leveraged, off your mobile phone. It’s a win win win – if I’m right. Apple gets the tech and talent to build the guts of its next interface, consumers get a better OS, and Twitter gets to keep its cash and eliminate a potential competitor to boot.

Smart move, Apple. I hope I’m right.

*For the record, I spoke to no one at Twitter or Apple before I wrote this. It’s all my own brand of pure speculation. 

Google Now: The Tip of A Very Long Spear

By - October 09, 2013

Yesterday my co-author and I traveled down to Google, a journey that for me has become something of a ritual. We met with the comms team for Google X, tested Google Glass, and took a spin in a self-driving car. And while those projects are fascinating and worthy of their own posts (or even chapters in the book), the most interesting meeting we had was with Johanna Wright, VP on the Android team responsible for Google Now.

Some of you might respond – “Google what?!” – and that’d be normal. Google Now is one of those products that to many users doesn’t seem like a product at all. It is instead the experience one has when you use the Google Search application on your Android or iPhone device (it’s consistently a top free app on the iTunes charts). You probably know it as Google search, but it’s far, far more than that. It’s the tip of a very important spear for Google, and if you study its architecture, all manner of interesting questions and insights can be found about where Google – and the Internet – may be headed.

When you fire up the Google search application on your phone, Google Now is all the bits that are not the familiar search bar. Here’s a screen shot of my Google Now “home page”:

gnow

As you can see, the search bar, which in a PC format is usually the *only* thing one sees, is most certainly not the main event. Certainly it’s at the top, and voice search is prominently featured (I could write 1,000 words just on voice search…another time, perhaps). But, the screen is dominated by “cards” of information – in this case a reminder of a call I have coming up (Google Now integrates with my calendar and contacts), as well as information about my drive home (Google Now knows I usually drive home in the afternoon). If I were to scroll down, more “cards” of information are shown, including local weather, points of interest, and sports scores (when the SF Giants were playing this past summer, I’d see scores – because Google Now knew I searched for “SF Giants scores” a lot).

These cards are extremely important to understanding where Google is heading with not only search, but with all of its various services (the card interface is now incorporated into Google’s “knowledge graph” search results, Google+, Gmail, and Google Maps, among many others). First, the cards “know” things about me – most critically my location, but also my search history, my calendar and contacts, my browsing history, key links in my Gmail, and more. They show up based on what interests and needs that Google believes will be most important to me. In essence, they are very tangible expressions of Google’s pivot from being a company that answers search queries, to being a company that anticipates your most important questions in real time, and answers them before you ask.

This, of course, has been the holy grail of tech  for some time – predating Google and even Microsoft. But now that rich data streams course constantly through the silicon veins of a very personal mobile device, that long-held vision is becoming reality.

In short, Now is Google’s attempt at becoming the real time interface to our lives – moving well beyond the siloed confines of “search” and into the far more ambitious world of “experience.” As in – every experience one has could well be lit by data delivered through Google Now.

Google knows that this moment – the moment of our lives becoming data – is happening now, and the company is very, very focused on seizing it.

If you doubt my hyperbole, I’d not be surprised, but I tend to test such hyperbole on multiple senior sources working deeply inside Google. To each I posited this question: “Is Google Now one of the most important products  at Google today?” Each answered emphatically yes.

To see why, consider this message, which popped up on my screen as I was preparing to write this post:

share daily commute

This is Google, asking me if I’d like to let selected people know where I am, in real time, during my daily commute. Of course, I can only share that status with people who are also Google+ users (no option to share on Facebook, Twitter, Foursquare, etc) – and that’s my point. First, questions like these are habituating us to the idea of sharing intimate information about ourselves with others, in real time. Second, a feature like this is *only* available to Google Now because of its integration with Google+ – one platform is reinforcing the other. Will Google let others play in this sandbox? Such a feature raises a very important question about what kind of world we want to live in – a world dominated by tightly integrated vertical platforms, or a world, as David Weinberger elegantly stated it, made up of small pieces loosely joined?

It was this question that weighed on my mind as I sat down with Johanna Wright yesterday. Since introducing Google Now (and the extremely related Google Knowledge Graph), the company has introduced more than 40 cards – cards for hotels, car rentals, and other travel information (like boarding passes), cards for movies, events, music and local businesses, cards tracking your activity (like walking, biking, etc.), and cards for nearby restaurants. There’s even a card that listens to your TV and tells you what music is playing.

Sound familiar? It should, because, to put it in words we can all understand: There’s an app for that. Or rather, there are apps for each of those. Let me list just a few of them, in order what what I laid out above: Hotel Tonight, Expedia, Lyft, Sidecar, Travelocity; Fandango, NetFlix, Hulu, iTunes, Spotify, Eventful, Yelp, Foursquare; Fitbit, Jawbone Up, Fuelband, Human; OpenTable, Urban Spoon; Shazam.

Google Now supplants the need to open an app by surfacing cards – cards that magically turn into just the information you need, when you need it – *without having to go to an app to get it.*

You following where this is going? Google is potentially disrupting the app world much the way its Universal Search disrupted major web properties  - taking the most valuable service or information, and surfacing it up for free. You may recall that universal search was quite controversial when it came out, because it appeared to favor surfacing Google-owned properties, such as YouTube, Finance, and Maps, over other web properties. Now, six years later, Universal search is, well universal, and that debate, which included an FTC investigation,  is over. Google properties won.

It’s worth noting that a key product manager for Universal Search was Johanna Wright, now the VP over Google Now. With all this in mind, I asked Wright about Google’s plans for Now: Would it be an open platform, where third parties can compete to be surfaced based on merit, or would favored services win out? And would various commercial products and services be able to pay to get integrated into Now’s suggestions and services?

Wright was understandably careful with her words when approaching this question. She declined to talk about monetization and business models for Now, but she did note that Google’s overall philosophy was one that favored the open web. The key, she said, was that Google get the user experience for Now right. The business model will come later (though she did note that Google Offers was already integrated into Now).

While Wright deferred comment on Now’s business model, I have no doubts there are plenty of folks inside Google thinking long and hard about the next steps the company will take to monetize Wright’s work. For now (no pun intended), Google Now is, in the main, a closed platform – surfacing only information that Google has deemed worthy of being surfaced, and integrating on a selective basis with only those services that Google believes will add value its consumers  (Google’s restaurant card, for example, integrates with OpenTable). Just as it did with search, Google is angling to control a key moment of a person’s daily life and attention – the point at which we lift our phone up to receive new information. When and if Google Now become ubiquitous, I can certainly imagine that the question of access and fairness will once again be raised. This movie, it seems, is fated to play out once again.

Search and Immortality

By - September 19, 2013

google.cover.inddFunny thing, there I was two days ago, at Google’s annual conference, watching Larry Page get asked questions so pliant in nature they couldn’t be called softballs. They were more like tee balls – little round interrogatives gingerly placed on a plastic column for Page to swat out into the crowd. Not that we would expect anything else – to be clear, this is Google’s event, and I see nothing wrong with Google scripting its own event. I had moderated the final session of the day, but Larry was the final speaker. Perhaps wisely, Google brought  someone else on to “grill” Page – those were his words as the interview started. (You be the judge –  a sample question: “What are your thoughts about tablets in schools?”)

Anyway, I was certainly not the right choice to talk to Larry. I know the folks at Google well, and have tons of respect for them. We both know I would have insisted on asking about a few things that were, well, in the news at the moment of that interview on Tuesday. Like, for example, the fact that Google, on the very next day, was going to announce the launch of Calico, a company seeking to solve that “moonshot” problem of aging. Oh, and by the way, current Apple Chair and former Genentech CEO Arthur Levinson was going to be CEO, reporting to Page. Seems like pretty interesting news, no? And yet, Larry kept mum about it during the interview. Wow. That’s some serious self control.

And yet I think I understand – each story has its own narrative, and this one needed room to breathe. You don’t want to break it inside an air-conditioned ballroom in front of your most important clients. You want to make sure it gets on the cover of Time (which it did), and that the news gets at least a few days to play through the media’s often tortured hype cycle. It’s grinding its way through that cycle now, and I’m sure we’ll see comparisons to everything from Kurzweil (who now works at Google) to Bladerunner, and beyond.

But what I was reminded of was the very end of my book on search, some 8 years ago. I was trying to put the meaning of search into context, and I found myself returning again and again to the concept of immortality.  This was my epilogue, which I offer here as perhaps some context for Google’s announcement this week:

“Search and Immortality”

On a fine sunny morning in 2003, not long after the birth of my third and most likely final child, I typed “immortality” into Google and hit the “I’m feeling lucky” button. I can’t explain why I turned to a search engine for metaphysical comfort, but I sensed the search might lead me somewhere—here I was writing a book about search, but what did it matter, really, in the larger scheme of things?

In an instant, Google took me to the Immortality Institute, an organization dedicated to “conquering the blight of involuntary death.”

Not quite what I was looking for. So I hit the search again, but this time I took a look at the first ten results, etched in blue, green, and black against Google’s eternal white.

Nothing really caught my eye. Cryonics stuff, a business called Immortality Inc., pretty much what you might expect. I couldn’t put what I was looking for into words, but I knew this wasn’t it.

Then I noticed the advertising relegated to the right side of the screen. There were four ads, each no more than three lines of text. The first was someone who claimed to have met immortal ETs. Pass. The third and fourth were from eBay and Yahoo Shopping. These megasites had purchased the immortality keyword in some odd and obliquely interesting hope that people searching for immortality might well find relief through . . . buying shit online. (In fact, what Yahoo and eBay were doing was the equivalent of search arbitrage— buying top positions for a search term on Google and then creating a link to the exact same search term on their own sites, in the hope of capturing high-value customers).

Interesting, but I wasn’t looking to buy the concept of immortality; I wanted to understand it. I took a pass on those as well. But the second paid link pointed to the epic Gilgamesh, which I hazily recalled as the first story ever written down—in Sumerian cuneiform, if memory served. I clicked on the link, earning Google a few pennies in the process, and landed on an obscure bookseller’s page. The epic of Gilgamesh, the site instructed me, recounts mankind’s “longing stretch toward the infinite” and its “reluctant embrace of the temporal. This is the eternal lot of mankind.”

Bingo. I didn’t quite know why, but this was the stuff I was looking for. My vague desire to understand the concept of immortality had brought me to the epic of Gilgamesh, and now I was hooked. My search was bearing fruit. But I didn’t want to buy a book and wait for it to come. I was in the moment of discovery, the heat of possible consummation. I wanted to read that epic, right now.1 So I typed the title itself into Google, and once again found myself larded with options.

But this time the organic results (the search results in the middle of a Google page, as opposed to the ads on the right) nailed it: the first two offered direct translations of the stone tablets upon which the epic is written. Clicking on the first link, I found a Washington State University professor’s summary of the Gilgamesh story. It read:

Gilgamesh was an historical king of Uruk in Babylonia, on the River Euphrates in modern Iraq; he lived about 2700 b.c. Although historians . . . tend to emphasize Hammurabi and his code of law, the civilizations of the Tigris-Euphrates area, among the first civilizations, focus rather on  Gilgamesh and the legends accruing around him to explain, as it were, themselves. Many stories and myths were written about Gilgamesh, some of which were written down about 2000 b.c. in the Sumerian language on clay tablets which still survive . . . written in the script known as cuneiform, which means “wedge-shaped.” The fullest surviving version, from which the summary here is taken, is derived from twelve stone tablets . . . found in the ruins of the library of Ashurbanipal, king of Assyria, 669–633 b.c., at Nineveh. The library was destroyed by the Persians in 612 b.c., and all the tablets are damaged. The tablets actually name an author, which is extremely rare in the ancient world, for this particular version of the story: Shin-eqi-unninni. You are being introduced here to the oldest known human author we can name by name!

In my search for immortality, I had found the oldest known named author in the history of Western civilization. Thanks to the speed, vastness, and evanescent power of Google, I came to know his name and his work within thirty seconds of proffering a vaguely worded query. This man, Shin-eqi-unninni, now lived in my own mind. Through his writings, with an assist from Google and a university professor, he had, in a sense, become immortal.

But wait! There’smore. Gilgamesh’s story is one of man’s struggle with the concept of immortality, and the story itself was nearly lost in an act of literary vandalism—the destruction of a great king’s library. As I contemplated all of this, sensing that, just possibly, I had found a way to explain why search was so important to our culture.

I read the first tablet’s opening lines:

The one who saw all (Sha nagba imuru) I will declare to the world, The one who knew all I will tell about [line missing] He saw the great Mystery, he knew the Hidden: He recovered the knowledge of all the times before the Flood. He journeyed beyond the distant, he journeyed beyond exhaustion, And then carved his story on stone.

What does it mean, I wondered, to become immortal through words pressed in clay—or, as was the case here, through words formed in bits and transferred over the Web? Is that not what every person longs for—what Odysseus chose over Kalypso’s nameless immortality— to die, but to be known forever? And does not search offer the same immortal imprint: is not existing forever in the indexes of Google and others the modern-day equivalent of carving our stories into stone? For anyone who has ever written his own name into a search box and anxiously awaited the results, I believe the answer is yes.

Something to think about, anyway. Good luck, Mr. Levinson and Mr. Page. I’m cheering you on, even if I can’t quite explain why. Maybe it’s that missing line from Gilgamesh we’re all trying to find….

*Hat tip to one of my editors Bill Brazell, for pinging me as I was writing this about this very news.

Hasta La Alta Vista, Baby.

By - June 29, 2013

I just saw the news that Yahoo! is “sunsetting” Alta Vista, one of the first “good” search engines. This makes me a little misty, as Alta Vista was the search engine I used BG – Before Google – and it had a real shot at *being* Google, had its various owners not utterly screwed it up over the years. Did you know, for example, that at one point Alta Vista was the largest and most widely used search tool on the web? Its driving force, Lois Monier, once told me “search should be a pencil” – he was adamant that Alta Vista not become a portal.

But Alta Vista was owned by DEC, a now dead computer company, which was bought by Compaq, another now dead computer company. And they made it a portal. And through the now defunct Overture, the assets of Alta Vista made their way to Yahoo!, a still alive portal. But now, Alta Vista is going to truly be dead.

It’s hard to watch an important player in the early Internet go away – and it makes me reflect on a couple of things. First, how much or our own culture and history we’re losing day by day, even despite the best efforts of archivists like Brewster Kahle. And secondly, on a personal note, Alta Vista was the search engine that helped me find my birth mother way back in 1995, when my wife was pregnant with our first child, and my life-long wonderings as an adoptee took on a new urgency. Alta Vista pointed me to an online forum for people like me, and there I found a person who helped me find my mother. Pretty cool.

So I’ll miss you, Alta Vista. And if any of you want to know the engine’s journey, well, there’s a book for that.

On Google Glass and OpenCo NYC

By - May 09, 2013

In case you have any interest, here’s a short clip of me opining on Google Glass and the upcoming OpenCoNYC, which is going to be HOT. More on that soon.

We’ve Seen This Movie Before…On Traffic of Good Intent

By - April 26, 2013

(image) Back in 2005 I whipped off a post with a title that has recently become relevant again – “Traffic of Good Intent.” That post keyed off  a major issue in the burgeoning search industry – click fraud. In the early days of search, click fraud was a huge problem (that link is from 2002!). Pundits (like me) claimed that because everyone was getting paid from fraud, it was “something of a whistling-past-the-graveyard issue for the entire (industry).” Cnet ran a story in 2004 identifying bad actors who created fake content, then ran robots over AdSense links on those pages. It blamed the open nature of the Web as fueling the fraudsters, and it noted that Google could not comment, because  it was in its quiet period before an IPO.

But once public, Google did respond, suing bad actors and posting extensive explanations of its anti-fraud practices. Conversely, a major fraud-based class action lawsuit was filed against all of the major search engines. Subsequent research suggested that as much as 30% of commercial clicks were fraudulent  - remember, this was after Google had gone public, and after the issue had been well-documented and endlessly discussed in the business and industry press. The major players in search finally banded together to fight the problem – understanding full well that without a united front and open communication, trust would never be established.

Think about that little history lesson – a massive, emerging new industry, one that was upending the entire marketing ecosystem, was operating under a constant cloud of “fraud” which may have been poisoning nearly a third of the revenues in the space. Yet billions in revenue and hundreds of billions in market value was still created. And after several years of lawsuits, negative press, and lord-knows-how-much-fraud, the clickfraud story has pretty much been forgotten.

Sound familiar?

It should. Because the same movie is once again playing, but this time the problem has migrated to the open ecosystem of programmatic display. As anyone who’s studied the LUMAscape knows, we now have a VC-fueled industry worth billions, with many players primed to go public in the coming year or so. And the original search players – Google in particular, but also Microsoft and Yahoo! – are also major actors in this new industry.

My post from January of this year - It’s Time To Call Out Fraud In The Adtech Ecosystem - summarized the new breed of fraud in our industry, and recently, many publications  have intensified their coverage of the topic. In late February, I invited a handful of adtech CEOs to a lunch where we discussed the issue, and everyone at the table – from AppNexus to Google, OpenX to MediaOcean – agreed that it was time to address the problem head on.

And that’s how we got to the news  this past week that the IAB is standing up a task force on “Traffic of Good Intent.” I’m proud to be a co-chair of the group (and yes, the name does come from that 2005 post in these pages). This time around, there are many more players, a much larger industry, and a far more complicated ecosystem. But it’s worth remembering that bad actors always take advantage of open systems. It’s up to us to unite and drive them back. We should all be trading in traffic of good intent – real human beings, engaged with real content and services across the Internet. Our customers, partners, investors, and our good company names depend on it.

I look forward to the work.

It’s Time To Call Out Fraud In The Adtech Ecosystem

By - January 26, 2013

A confusing landscape = ripe opportunities for fraud.

As part of research I’m doing both for the book and for my upcoming conference (the CM Summit, more on that soon), I’ve been in pretty extensive conversations lately with dozens of key players in the advertising technology industry. I find the ecosystem that has developed  to be fascinating, complex, and ripe with opportunity (and deeply important to the future of our society, not just marketing). I’ll be writing about it quite a bit in coming months. But before I do, I wanted to call out a growing issue that our industry will have to tackle sooner rather than later.

Just as in the early, wild west days of search (1999-2004), the programmatic advertising business – a multi-billion dollar marketplace growing faster than search, video, or anything else for that matter – is riddled with fraud.

That’s what many very reputable sources have told me in great length over the past few months. It’s something of an open secret, and more and more people are speaking out against it. Here’s Federated Media’s Walter Knapp on the problem, back in March of last year:

The great thing about the Internet is that it is built on the foundation of openness — from the way the domain system works to the way content and publishing are increasingly democratic. The core technologies embrace openness, sharing, linking and the ability to consume content across devices and across wired or wireless connections. Unfortunately, the openness we depend on in the digital media business is also available to people who can (and will) take advantage of this openness and exploit it for their own selfish wants.

Knapp notes two forms of fraud – ad injectors, fraudulent browser plugins that take over ad calls; and the practice of inserting an entire site into a 1×1 pixel hidden on high traffic but low quality sites featuring porn or music lyrics. Both are examples I’ve heard about over and over in my reporting. A third involves “stacking” ads one behind the other, all playing video to completion, often playing in inactive tabs. A fourth features refreshing ad calls on accelerated schedules or in inactive tabs. Yet another involves running as many ads as possible out of view, simply to gain “view through attribution” on a closed loop success metric.

More people are starting to call these practices out. AppNexus CEO Brian O’Kelly prominently featured the issue of fraud in his blog post celebrating his company’s recent $75 million funding, and what he intends to use it for:

Quality We will continue to invest in cleaning up the advertising marketplace. We’re proud of our anti-piracy stance, and our 5x volume growth this year indicates that you don’t need to serve on BitTorrent sites to be an ad platform company. We are investing heavily in fighting fraud, porn, malvertising, and malicious toolbars, and we are actively working on viewability tools.

Programmatic industry watcher AdExchanger puts it this way:

AppNexus’s pledge to invest money in ad quality issues is worth calling out. The issue is becoming more pervasive as companies emerge to exploit the vulnerabilities of real-time traded inventory to data and impression fraud, malvertising, and other nefarious practices. Fraudulent activities aside, the emergence of robust ad verification and viewability tools means display ad marketplaces and buying platforms must keep a clean nose.

It’s true that many folks are working on addressing the issue, including the IAB. But the bad actors are currently far ahead of the good guys, and worse, many in our industry are turning a blind eye, hoping the problem goes away in time, without too much publicity. Why? Well, nearly everyone gets paid from fraud – the publishers, the exchanges, the data providers, and the agencies. Even the marketers,who are footing the bill, feel like they are getting value – because the success metrics they’ve set up are being  met.

But fraud hurts the ecosystem in a massive way. It means that low quality, invisible, or purely fraudulent inventory is holding down the average value of the entire marketplace – hurting high quality, engaged publishers in the process, stunting investment in quality content.

Over and over, I hear that the reason CPMs (the amount of money a marketer is willing to pay for one thousand advertising impresssions) are so low is because “there’s infinite inventory.”

Hogwash. There’s only so much time in the day, and only so many pages where actual human beings are really paying attention, and the web (including mobile) is growing at a finite pace. There are even fewer places where marketers can be assured of quality, engagement, and appropriate context. It’s time we focus on identifying them, and ridding ourselves of the true source of “infinite inventory” – fraud.

Portrait of Twitter As A Young Media Company

By - January 21, 2013

Last year I predicted that Twitter would become a media company. However, I focused mainly on the new “Discover” functionality, and I probably should have gone a lot further. In this piece, I intend to.

So I’ll start with this: 2013 will be the year Twitter starts to create, curate, and co-create media experiences on top of its platform. I hinted at this in my brief coverage of Twitter’s Oscar Index (see Twitter’s Makin’ Media), but allow me to put a bit more flesh on the bones.

So what might one make from the fact that your platform captures hundreds of millions of individuals declaring what’s going on at any give time? Well, let’s break down some of the signals in all that supposed noise. As I’ve written over and over and over in the past several years, Twitter presents a massive search problem/opportunity. For example, Twitter’s gotten better and better at what’s called “entity extraction” – identifying a person, place, or thing, then associating behaviors and attributes around that thing. This (among other reasons) is why its Discover feature keeps getting better and better. Another important signal is location – Twitter is increasingly focused on getting us to geolocate our tweets. A third signal is the actual person tweeting – his or her influence and interest graph. Yet another signal is time – when was the entity tweeted about?

Real time entity extraction crossed with signals like those described above is the Holy Grail – and I’m guessing Twitter is almost, if not already there.

Once you get good at all these things (and more), a number of really interesting possibilities open up. Identifying “big things” that are going on at any given time is something that Twitter already does – though not particularly well (the best window in is the “Trends” box on the left of the page). Regardless, Twitter has become a go-to service for quick updates about news events (Sandy, Newtown, etc), entertainment events (SuperBowl, Oscars, Grammys, etc), and well….pretty much any kind of event.

But so far, it’s not exactly easy to get the big picture of what’s really going on for any given event on Twitter. In fact, it’s rather difficult. You can search for a hashtag, or keywords you think are associated with an event, but no matter what, it’s extremely difficult to makes sense of it all. For a big event like Sandy Hook or the Oscars, there are literally millions of tweets to sift through. And those tweets have millions of pictures, links, and videos. How can you know what’s important?

This is exactly the problem that  media experiences are designed to solve. By combining intelligent algorithms (these tweets are retweeted more than others, this video is linked to more than all the others, etc) and some smart editors, Twitter can (and most likely will) surface instant windows into events as they unfold around the globe. I imagine logging into Twitter at some point in the future and seeing a dashboard not of Trends, but of “Happenings” – Events edited to my interest graph, location, and the like. When I click on on of those events, I enter a meticulously edited media experience – a pulsing, ever changing feast of information tailored around that event.

So, put in one sentance: Twitter’s going to do events soon.

What other media experiences might Twitter create? Well, extending the logic, it only makes sense that Twitter will curate media services, just as LinkedIn and now Facebook are starting to do (I argue that Graph Search is a media play here).

“Just Landed” – from 2009.

As Google has proven, words have a lot of power on the web. They have even more power when put in context at scale. Consider what happened when a data artist asked a simple question: Where are people when then tweet that they “just landed”?

Now, imagine Twitter stands up a service that allows you to see patterns around phrases like “looking for someone to…,” or “just got a job,” or “python developer,” etc. Yep, lurking inside all that Twitter data is a pretty powerful job service. And I’m only using jobs as a straw man (and because it’s a driving force of LinkedIn’s success, of course). When you have humanity whispering into your ear at scale, you can tune in any number of valuable signals. Getting a job is one important signal. But so is getting married, buying a house or a car, graduating, and, and and….well you get the picture. Standing up “media services” around these life milestones is what media companies do. They used to be called magazines. What might Twitter call them? In 2013, we’ll most likely find out.

So far I’ve proposed two new media features of Twitter: Events and Media Services. I’ll round out this post with a prediction around a third: Video. Video is a vastly under-leveraged asset on Twitter, but people are sharing millions of links to video clips every day on the service. I imagine that Twitter will soon offer some kind of video curation feature – giving its base the ability to find the most popular videos based on pivot points of time, interest, and people. Surfacing and creating more video on the Twitter service has got to be a major priority at the company. And let’s not forget that Twitter bought Vine, after all…

After all, everybody loves video. In particular, advertisers love video. After all, Twitter is already working with Neilsen to become the official barometer of television conversations.

Which brings me to the “stick the landing” portion of this particular round up. Twitter is going to make much more media this year, because Twitter is going to make much more money this year. Each of the features I described above – Events, Media Services, and Video – bring with them inherent business models. I don’t expect they’ll look like traditional display models, of course, but I would not be surprised if they strayed a bit from Twitter’s current Promoted Suite products. With new media products come new advertising products. And new revenue.

Time will tell if I got this one right. Meanwhile, what do you think?