free html hit counter Book Related Archives | Page 7 of 30 | John Battelle's Search Blog

The debut of “else”: Surveillance Everywhere and the Technological Wild West

By - August 13, 2013

As we’re working on the book, Sara and I are planning on sharing some of the news items and blog posts that catch our attention each week. We’ll also plan on talking through some of the things we’re reading and working on in this space. In keeping with boolean condition logic of the if/then working title for the book, we’ll be tagging these posts as “else.” Links aren’t necessarily endorsements, but they do point to ideas that got us thinking this week.

If you want to keep up with what we’re reading/thinking about on a weekly basis, the best way is to subscribe to the ELSE feed, either as an email newsletter (coming soon) or through RSS.

—–

This week we look at challenges in the using quantified self data, developments in the NSA surveillance coverage, and round out with a few throwbacks to the Victorian age of technology. On to the links:

Inside the Nike+ Accelerator: Fueling the quantified-self movement – Engadget

TechStars accelerator working on Nike+ to build innovation on the Nike platform. Article talks a lot about the importance of opening up the Nike+ API for development innovation (right now it’s only open to these ten accelerator companies).

Why The Quantified Self Needs A Monopoly – ReadWrite

Highlights one of the big barriers to consumer adoption right now, that is correlating all these quantified self data sources into one, meaningful view. To do that you need to a) be able to get the data into one place, b) have it speak to each other, and c) know what you are looking at once you can see it all in one place. We might argue that you don’t need an Apple or Microsoft monopoly for that necessarily. But we will need tools that pull this together; maybe something more along the lines of Mint.

The Public-Private Surveillance Partnership – Bloomberg

Bruce Schneier walks us through the implicit business models that got us into the current surveillance state: “Imagine the government passed a law requiring all citizens to carry a tracking device. Such a law would immediately be found unconstitutional. Yet we all carry mobile phones.” [Incidentally, Schneier is also a Fellow at the Berkman Center this year along with Sara].

This Recycling Bin Is Stalking You - The Atlantic Cities

Recycling bins in London are tracking MAC addresses from passing smartphones and Wifi-enabled devices, essentially bringing tracking cookies from the internet into the physical world. Turns out this might actually be illegal.

The Atlantic Cities

 

A Cheap Spying Tool With a High Creepy Factor – NYTimes

Security researcher Brendan O’Connor uses cheap Raspberry Pi devices to monitor Wifi signals, proving that conducting surveillance is becoming easier, no matter who you are.

Other Agencies Clamor for Data N.S.A. Compiles – NYTimes

Once the data is in a format where it can be activated, others will find new uses for it.

Searching Big Data for ‘Digital Smoke Signals’ – NYTimes

The United Nations Global Pulse team is using sentiment analysis and mobile data to catch early signals for global economic trends to develop faster, more adaptive and responsive aid programs.

Why the Sun is Setting on the Wild West of Ride-Sharing – Wired

Car sharing dropoffs at airports are started to see a crackdown in SFO. Policies still protect taxi and limo service domain here and new regulations requiring insurance companies could increase operating costs. This could slow down the markets where consumers are taking underused assets and making them liquid. John recently wrote about how Uber saved the day in a recent travel snafu.

3-D Printing the 19th Century – NYTimes

Martin Galese is bringing back patents from a bygone era, 3-D printing them in all their beautifully-designed glory. Some of these designs might not have been easily manufactured in their time.

Last week Sara was reading Rebecca Solnit’s River of Shadows: Eadweard Muybridge and the Technological Wild West. The book looks at the historical context around Muybridge’s photographic technology developments that increased the shutter speeds and introduced the ability to almost slow down time into smaller knowable bits. These developments paved the way for modern cinema, but also ran parallel to Victorian explorations of scientific discovery. Sara wrote about some interesting parallels with Muybridge’s body movement studies and the Quantified Self movement; film allowed us to slow down and dissect the bodies’ gate; sensors like the Fitbit allow us to track a walking gate all day long.

  • Content Marquee

Four Minutes on the Future of Marketing

By - August 05, 2013

Earlier this year I sat down with a videographer at the Bazaarvoice Summit in Austin. He asked me about the future of marketing, in particular as it related to data and consumer behavior. Given what I announced earlier this morning, I thought you might find this short video worth a view. Thanks to Ian Greenleigh for doing all the work!

Liquid Markets Win: On Boats, Planes and Automobiles

By - July 30, 2013

Marthas_1775_Blask-wbOne of the key themes in our upcoming book has to do with the interaction of information and the physical world – in particular, how all things physical become “liquid” when activated by just the right information. But when you’re writing (and thinking out loud) about this topic, it’s easy to fall into an academic cadence, because information theory is a thicket – just try reading “The Information” in one sitting, for example.

So I’ve found it’s best to just tell stories instead (and to be honest, I’d wager that nearly all information theory should be reduced to narrative, because narrative is how we as humans make sense of information, but I digress). Here’s a story that happened just this past weekend.

If you’ve been reading for more than a year, you know that I spend a good part of August working on an island off the coast of Massachusetts. It’s a special place where my great grandmother settled in the 19th century, the kind of place where you visit graveyards with your kids to remind them of their own history, then hit a carousel and ice cream shop in the afternoon.

Anyway, this year my time on the island is unfortunately brief, what with my coming back to FM and various other entanglements. So every day “on island” is a precious one. Last week my son and I went on an East Coast college tour, driving from Washington DC (Georgetown, American) through Pennsylvania (Bucknell, etc) to Boston (Northeastern, BU, etc). After touring MIT on Friday, we decided to come to the Island a day early, on Sat. That way we could open up the rental house, get the car, and prepare for the rest of our family – my wife and two daughters –  who were flying in on Sunday.

On Sunday my son and I were settled into our rental, eagerly awaiting Mom and sisters’ arrival. They landed in Boston just fine, but hit a major hitch with their connection on a tiny airline called Cape Air. Now Cape Air’s largest plane seats about 8 people, as they specialize in one thing – hopping from Boston down to the Islands. Apparently some cross winds and rain fouled up the routes, and long story short, my family’s flight was delayed for two hours, then cancelled altogether (oh, and they lost my wife’s luggage too). By then, it was almost 9pm, and too late to drive the 80 miles down from Boston to the ferry in Falmouth, which is the only other way to get to the island. (The last ferry leaves at 9.45 pm).

We were all distraught – there they were, just 80 miles away, but with no way to get to us. We were going to lose one of our precious days on the island, and it just stunk.

Then a fellow stranded passenger mentioned a possible solution to my wife: There was a service in Falmouth that aggregated private commercial boats for use as water taxis. Maybe they’d be able to help?

My wife mentioned this to me, and sure enough, a quick Google search found them. At 8.30 pm I called the service and “Captain Jim” hooked me up with another lobster boat captain ready to take my family across the sound, even late at night. Awesome!

But we still needed to get the girls that 80 miles down to Falmouth, and it was past “business hours.” I’ve used a lot of car services in my business life, and I know that they are not exactly very flexible – you have to make a reservation well in advance, and they cost a lot of money for long drives. I wasn’t expecting much, but I started calling as many as I could find, asking if they had any cars near Boston’s Logan airport *right now*.

The car services I called acted exactly as one might expect. Two put me on interminable hold while the “checked to see if they had a car near the airport.” A third flat out refused to try. A fourth asked me to put the girls into a taxi and send them back into downtown Boston, where they might have a car. And so on.

That’s when I tweeted this out:

 

And this is when the story starts to get interesting, in terms of liquid, information-driven markets interacting with the physical world. It turns out, Uber *does* have the ability for someone to drop a pin remotely, I just didn’t know it – I didn’t have *the information* I needed. Twitter solved that in an instant, as one of my followers quickly clued me in about how to do it. In two minutes I was called by an Uber driver at the airport, who was ready to whisk my family down to the waiting lobster boat. An hour and a half later, I picked them up on a private dock near the house. What the private boat service and Uber did was take inactive, physical objects, in this case a Lincoln towncar and a fishing boat, and turn them into kinetic, liquid, real-time addressable assets. And the main reason this was possible? Information cycling through a digital foundation of cell phone towers, Google, and apps like Uber.

What’s even more interesting about this story are the economics: The cost to fly three people via Cape Air to the Island was nearly 25% *more* than taking Uber and the chartered water taxi. Add to this the fact that the Uber driver was far more friendly and eager to please than your typical car service guy, and the water taxi was both fun and nearly twice as fast as the ferry.

In short, having the a liquid, information-driven market of cars and boats created a cheaper, faster, and way more enjoyable experience for my family. That’s a great thing, to my mind, and it makes me optimistic about the coming liquid economy. But if I operated Cape Air or Carey Limousine, I’d be more than nervous right about now….

 

An Afternoon at the Media Lab: Where The Lines Between Physical and Digital Are Permeable

By - July 29, 2013

IcanHasHOloYesterday I took my son to the MIT Media Lab, hallowed ground for me, as reading Stewart Brand’s 1988 “The Media Lab” propelled me toward helping to create Wired magazine, where I edited the founding Director of the Lab, Nicholas Negroponte, for five years (he wrote the back column of the magazine).

For this visit, I met up with David Kong, one of the lab’s alumni wizards, who took us on a whirlwind tour of the place (David’s work on microfluidics is, I believe, some of the most important stuff being done today, but more on that in another post). I spent a day there last summer with Director Joi Ito, and it’s amazing to see how much progress can be made in a year.

Instead of describing everything, I think I’ll let video do the work – one of the Lab’s core values is to always be demo’ing, and my son and I saw half a dozen incredible projects, all demo’d by the people who created them.

First up was the Opera of the Future group, which evolved out of the HyperInstruments lab. Akito Van Troyer gave us a tour of the components used in the recently staged “Death and the Powers” piece, which was a finalist for the Pulitzer in music.  Here’s some video of that performance:

Akito also turned us onto a very cool beat machine he built as a side project, a hack based on actuators and tempo that turns anything into a percussion instrument. Here’s a video of that I found on YouTube:

After that we went to see Xiao Xiao, who works in the Lab’s Tangible Media group (this is the part of the Lab most directly connected to the themes of the upcoming book). She showed us the MirrorFugue, which is just amazing, in particular, to sit down at the keyboard as it’s playing. It’s magical, which is pretty much the goal of the entire Lab. Here’s a video of that:

MirrorFugue III from Xiao Xiao on Vimeo.

You can probably sense a theme by now – all this work is about blurring the lines between physical and digital, atoms and bits. An extraordinary world is soon to be settled by pioneers in this space, and we’re all of us fascinated by it – it’s why we love the idea (if not necessarily the look) of Google Glass, or 3D printing (I met the co-founder of FormLabs while at the Lab), or cool gadgets like the NFC ring.

The Media Lab is a place where folks are actively creating the future. Over and over, I heard this refrain: “I took some off the shelf parts, hacked them together, and wrote some code.” Simple, right?

One example: Makey Makey, which went viral earlier this year with the “banana piano.” The idea is bigger than turning fruit into keyboards, however. It’s about making nearly anything physical a portal into the digital world, and bringing the digital right back into the physical. I met with Eric Rosenbaum, one of the creators, in his lab, which is called “Lifelong Kindergarten” (yeah, I know.) Here’s a short video about Makey Makey:

As the border between physical and digital gets more permeable, a new kind of literacy emerges. And that literacy is built on a foundation of code – whether it’s the codes of letters and words, or the code of bits and algorithms. Rosenbaum showed me Scratch, a graphical programming language used by hundreds of thousands of kids across the world. I’m determined to learn how to code, at least enough to be dangerous (I took classes in Pascal about 30 years ago…). Maybe Scratch is where I’ll start.

Next up I met Dan Novy, from the Lab’s Object-Based Media Group. He showed us a number of great projects he’s working on, including holo-presence (with a sense of humor, see photo at top) and new forms of augmented experience. Check out this video about redefining the home entertainment experience:

Dan also took us into a small room with a voice aware projection device in the center. Using his voice, Dan told a children’s story, and the four walls of the room lit up with visual images related to the storybook. It’s early days, but we discussed what might happen when this device is miniaturized and connected to consumer “narrative catchers” like Facebook, Path, Google Glass, and the like. Also next to the projector was an object – what it is, it doesn’t matter, but for this example it was a bottle of perfume – and when you pick up that object, “memories” related to that object are projected onto the walls. So imagine what might happen when you pick up that ornament from Christmas three years ago and hang it on the tree, and images from that Christmas past flash onto your home’s walls….

The last demo we saw was perhaps the most well known of Dan’s group’s work. In essence, they turned a basketball net into a data collection device, so as to measure the force of a slam dunk. The technology is amazing, watch Dan talk about it here:

The Media Lab is truly an extraordinary place, and seeing it with my son made it even more magical. I’ve toured it a few times now, but I’ll never tire of coming back. The work happening there is helping to define the world all our kids will be living in soon.

The PRISMner’s Dilemma

By - July 16, 2013

prismSometimes when you aren’t sure what you have to say about something, you should just start talking about it. That’s how I feel about the evolving PRISM story – it’s so damn big, I don’t feel like I’ve quite gotten my head around it. Then again, I realize I’ve been thinking about this stuff for more than two decades – I assigned and edited a story about massive government data overreach in the first issue of Wired, for God’s sake, and we’re having our 20th anniversary party this Saturday. Shit howdy, back then I felt like I was pissing into the wind –  was I just a 27-year-old conspiracy theorist?

Um, no. We were just a bit ahead of ourselves at Wired back in the day.  Now, it feels like we’re in the middle of a hurricane. Just today I spoke to a senior executive at a Very Large Internet Company who complained about spending way too much time dealing with PRISM. Microsoft just posted a missive which said, in essence, “We think this sucks and we sure wish the US government would get its shit together.” I can only imagine the war rooms at Facebook, Amazon, Google, Twitter, and other major Internet companies – PRISM is putting them directly at odds with the very currency of their business: Consumer trust.

And I’m fucking thrilled about this all. Because finally, the core issue of data rights is coming to the fore of societal conversation. Here’s what I wrote about the issue back in 2005, in The Search:

The fact is, massive storehouses of personally identifiable information now exist. But our culture has yet to truly grasp the implications of all that information, much less protect itself from potential misuse….

Do you trust the companies you interact with to never read your mail, or never to examine your clickstream without your permission? More to the point, do you trust them to never turn that information over to someone else who might want it—for example, the government? If your answer is yes (and certainly, given the trade-offs of not using the service at all, it’s a reasonable answer), you owe it to yourself to at least read up on the USA PATRIOT Act, a federal law enacted in the wake of the 9/11 tragedy.

I then go into the details of PATRIOT, which has only strengthened since 2005, and conclude:

One might argue that while the PATRIOT Act is scary, in times of war citizens must always be willing to balance civil liberties with national security. Most of us might be willing to agree to such a framework in a presearch world, but the implications of such broad government authority are chilling given the world in which we now live—a world where our every digital track, once lost in the blowing dust of a presearch world, can now be tagged, recorded, and held in the amber of a perpetual index.

So here we are, having the conversation at long last. I plan to start posting about it more, in particular now that my co-author Sara M. Watson is about to graduate from Oxford and join the Berkman Center at Harvard (damn, I keep good company.).

I’ve got so many posts brewing in me about all of this. But I wanted to end this one with another longish excerpt from my last book, one I think encapsulates the issues major Internet platforms are facing now that programs like PRISM have become the focal point of a contentious global conversation.

In early 2005, I sat down with Sergey Brin and asked what he thinks of the PATRIOT Act, and whether Google has a stance on its implications. His response: “I have not read the PATRIOT Act.” I explain the various issues at hand, and Brin listens carefully. “I think some of these concerns are overstated,” he begins. “There has never been an incident that I am aware of where any search company, or Google for that matter, has somehow divulged information about a searcher.” I remind him that had there been such a case, he would be legally required to answer in just this way. That stops him for a moment, as he realizes that his very answer, which I believe was in earnest, could be taken as evasive. If Google had indeed been required to give information over to the government, certainly he would not be able to tell either the suspect or an inquiring journalist. He then continues. “At the very least, [the government] ought to give you a sense of the nature of the request,” he said. “But I don’t view this as a realistic issue, personally. If it became a problem, we could change our policy on it.”

It’s Officially Now A Problem, Sergey. But it turns out, it’s not so easy to just change policy.

I can’t wait to watch this unfold. It’s about time we leaned in, so to speak.

Hasta La Alta Vista, Baby.

By - June 29, 2013

I just saw the news that Yahoo! is “sunsetting” Alta Vista, one of the first “good” search engines. This makes me a little misty, as Alta Vista was the search engine I used BG – Before Google – and it had a real shot at *being* Google, had its various owners not utterly screwed it up over the years. Did you know, for example, that at one point Alta Vista was the largest and most widely used search tool on the web? Its driving force, Lois Monier, once told me “search should be a pencil” – he was adamant that Alta Vista not become a portal.

But Alta Vista was owned by DEC, a now dead computer company, which was bought by Compaq, another now dead computer company. And they made it a portal. And through the now defunct Overture, the assets of Alta Vista made their way to Yahoo!, a still alive portal. But now, Alta Vista is going to truly be dead.

It’s hard to watch an important player in the early Internet go away – and it makes me reflect on a couple of things. First, how much or our own culture and history we’re losing day by day, even despite the best efforts of archivists like Brewster Kahle. And secondly, on a personal note, Alta Vista was the search engine that helped me find my birth mother way back in 1995, when my wife was pregnant with our first child, and my life-long wonderings as an adoptee took on a new urgency. Alta Vista pointed me to an online forum for people like me, and there I found a person who helped me find my mother. Pretty cool.

So I’ll miss you, Alta Vista. And if any of you want to know the engine’s journey, well, there’s a book for that.

The Book Lives On

By - June 09, 2013

Faithful readers will recall that about three months ago, I announced my return to FM as CEO. I also mentioned that the projects I’d been working on – notably OpenCo and The Book, would have to be retooled given my new commitment to the company I started back in 2005 (when I last wrote a book). In the post, I wrote:

I love the book I’m working on, and I don’t plan to abandon it (I’m bringing on a co-author). And I love the conferences I do, and I’ll still be doing them (though I’ll be hiring someone to run them full time). But my first love is the company I started in 2005, whose story is not only unfinished, it’s at the height of its running narrative.

I’m very, very pleased to announce that I’ve found that co-author – her name is Sara M. Watson, and she’s simply the perfect partner for me to be working with on this book. You can read her post about it here. Sara and I met over Twitter, after she noticed the theme of the CM Summit – “Bridging Data and Humanity.” We spoke on the phone and I learned that the intersection of society and data was her passion – and that her background was an awful lot like mine. She started her career as a liberal arts major from Harvard (during the time Facebook was just a dorm room project), toiled in the narrative fields of enterprise IT, became fascinated with the story of information, and decided to head to graduate school to study it (she’ll finish her Masters from Oxford in a few months). After Oxford, Sara has some amazing plans lined up (I can’t talk about them yet) that dovetail perfectly into our shared work.

I started my career as a liberal arts major from Berkeley, wrote about enterprise IT for a few years, then followed my passion for the digital narrative into graduate school as well (also at Berkeley, the Oxford of the West, or perhaps, the Harvard – sorry Stanford!). My first project out of grad school was Wired magazine. Sara’s is going to be our book. I’m honored to be working with her. Last week in London I got to meet her for the first time and spend some quality time together.

The past 12 weeks have been a whirlwind, as I’ve gotten my arms around Federated, executed four conferences in New York, Cincinnati, and London, and lucked into finding great partners for the projects I’m passionate about. Not only have I found the perfect collaborator in Sara, I’ve also found a CEO to run OpenCo, which recently had an amazing London pilot and a successful debut in New York as well. But more about him later. For this post, I want to welcome Sara to the Searchblog community, and I expect the our partnership will result in a lot more writing coming through this channel in the near future.

Here’s a video of me talking about the themes of the book, and announcing Sara as well, at Le Web last week.

Onwards!

 

Behind the Banner, A Visualization of the Adtech Ecosystem

By - May 13, 2013

I’m very proud to announce “Behind the Banner“, a visualization I’ve been producing with Jer Thorp and his team from The Office for Creative Research, underwritten by Adobe as part of the upcoming CM Summit next week. You can read more about it in this release, but the real story of this project starts with my own quest to understand the world of programmatic trading of advertising inventory – a world that at times feels rather like a hot mess, and at other times, like the future of not only all media, but all data-driven experiences we’ll have as a society, period.

I’m a fan of Terry Kawaja and his Lumascapes – Terry was an advisory to us as we iterated this project. But I’ve always been a bit mystified by those diagrams – you have to be pretty well steeped in the world of adtech to grok how all those companies work together. My goal with Behind the Banner was to demystify the 200 or so milliseconds driving each ad impression – to break down the steps, identify the players, make it a living thing. I think this first crack goes a long way toward doing that – like every producer, I’m not entirely satisfied with it, but damn, it’s the best thing I’ve seen out there so far.

I am deeply grateful to all the folks who helped us make this happen, in particular Jared Cook at Adobe, and a legion of leaders in the industry who reviewed early versions, including Walter Knapp, Bill Demas, Ned Brody, Brian O’Kelley, Ann Lewnes, and dozens more who helped me research and imagine what this might end up looking like.

So take a look and tell me what you think. It’s far too complex to embed here, so we have it running over on the CM Summit site. If nothing else, it should get folks talking, and I hope you’ll help us make it better by leaving a comment here, or sending me mail with your thoughts.

Oh, and while you are at the site, check out the conference lineup. We are almost sold out of tickets, and it’s going to be one heckuva conversation, so please join us!

The iWatch: What I Hope Apple Actually Does (But Probably Won’t)

By - February 21, 2013

(image AppleInsider) Back in April of last year, I pondered Pebble, the then-wildly successful darling of Kickstarter fame. Pebble is a wristwatch device that connects to iPhones and displays various smart things. In the piece, Does the Pebble Cause a Ripple In Apple’s Waters?, I asked whether Apple would allow such third-party hardware to play in their backyard. It struck me Apple’s entire business was about hardware. Pebble, I figured, was in for a tough road. No wonder it went to Kickstarter, I mused. VCs would never back something so clearly in Apple’s target zone. From the post:

If you watch the video explaining Pebble, it become pretty clear that the watch is, in essence, a new form factor for the iPhone. It’s smaller, it’s more use-case defined, but that’s what it is: A smaller mirror of your iPhone, strapped to you wrist. Pebble uses bluetooth connectivity to access the iPhone’s native capabilities, and then displays data, apps, and services on its high-resolution e-paper screen. It even has its own “app store” and (upcoming) SDK/API so people can write native apps to the device.

In short, Pebble is an iPhone for your wrist. And Apple doesn’t own it.

If we’ve learned anything about Apple over the years, it’s that Apple is driven by its hardware business. It makes its profits by selling hardware – and it’s built a beautiful closed software ecosystem to insure those hardware sales. Pebble forces an interesting question: Does Apple care about new form factors for hardware? Or is it content to build out just the “core” hardware platform, and allow anyone to innovate in new hardware instances? Would Apple be cool with someone building, say, a larger form factor of the iPhone, perhaps tablet-sized, driven by your iPhone?

Fast forward to now. The month’s Apple rumors have all been about the “iWatch” – the company’s next big innovation.  Apparently reliable sources – most likely now muted thanks to Apple’s exceptional PR machine – have said that 100 people are working on the device inside Apple’s HQ. And this week came news that Apple has even filed for a patent around the concept. 

If I’m Pebble, I’m not sleeping well at night.

I have no idea if Apple will actually create such a device – though I’m certain it must be testing one.

However, if Apple really wants the device to take off, the company should incorporate more than just iPhone connectivity. Here’s my wish list:

- Open platform for connectivity. Any device can connect to the device, not just iOS. I know this is wishful thinking, but…for example, Google has opted for glasses as its next big thing in wearable computing. I certainly would like the two to work together. (And how cool would it be if it worked with Android? OK, sorry. Just had to ask.)

- Sensors and software that make the device the equivalent of the Fitbit or the Jawbone Up.

- Integration with those apps, so that users don’t lose their data if they want to move to Apple’s hardware platform.

- As with Pebble, an open app ecosystem for the device, not one locked down into iOS. (I know…)

- A warranty on breakage. It’s one thing to ignore the criminal cracking that happens with nearly every iPhone in existence, because you can blame the consumer for dropping the damn thing. But if this thing is on somebody’s wrist, it’s going to get smacked around. And if Apple takes the same approach to breakage as it has for the iPhone, the device will be a failure.

That’s my major wish list. What would you want from the device?

It’s Time To Call Out Fraud In The Adtech Ecosystem

By - January 26, 2013

A confusing landscape = ripe opportunities for fraud.

As part of research I’m doing both for the book and for my upcoming conference (the CM Summit, more on that soon), I’ve been in pretty extensive conversations lately with dozens of key players in the advertising technology industry. I find the ecosystem that has developed  to be fascinating, complex, and ripe with opportunity (and deeply important to the future of our society, not just marketing). I’ll be writing about it quite a bit in coming months. But before I do, I wanted to call out a growing issue that our industry will have to tackle sooner rather than later.

Just as in the early, wild west days of search (1999-2004), the programmatic advertising business – a multi-billion dollar marketplace growing faster than search, video, or anything else for that matter – is riddled with fraud.

That’s what many very reputable sources have told me in great length over the past few months. It’s something of an open secret, and more and more people are speaking out against it. Here’s Federated Media’s Walter Knapp on the problem, back in March of last year:

The great thing about the Internet is that it is built on the foundation of openness — from the way the domain system works to the way content and publishing are increasingly democratic. The core technologies embrace openness, sharing, linking and the ability to consume content across devices and across wired or wireless connections. Unfortunately, the openness we depend on in the digital media business is also available to people who can (and will) take advantage of this openness and exploit it for their own selfish wants.

Knapp notes two forms of fraud – ad injectors, fraudulent browser plugins that take over ad calls; and the practice of inserting an entire site into a 1×1 pixel hidden on high traffic but low quality sites featuring porn or music lyrics. Both are examples I’ve heard about over and over in my reporting. A third involves “stacking” ads one behind the other, all playing video to completion, often playing in inactive tabs. A fourth features refreshing ad calls on accelerated schedules or in inactive tabs. Yet another involves running as many ads as possible out of view, simply to gain “view through attribution” on a closed loop success metric.

More people are starting to call these practices out. AppNexus CEO Brian O’Kelly prominently featured the issue of fraud in his blog post celebrating his company’s recent $75 million funding, and what he intends to use it for:

Quality We will continue to invest in cleaning up the advertising marketplace. We’re proud of our anti-piracy stance, and our 5x volume growth this year indicates that you don’t need to serve on BitTorrent sites to be an ad platform company. We are investing heavily in fighting fraud, porn, malvertising, and malicious toolbars, and we are actively working on viewability tools.

Programmatic industry watcher AdExchanger puts it this way:

AppNexus’s pledge to invest money in ad quality issues is worth calling out. The issue is becoming more pervasive as companies emerge to exploit the vulnerabilities of real-time traded inventory to data and impression fraud, malvertising, and other nefarious practices. Fraudulent activities aside, the emergence of robust ad verification and viewability tools means display ad marketplaces and buying platforms must keep a clean nose.

It’s true that many folks are working on addressing the issue, including the IAB. But the bad actors are currently far ahead of the good guys, and worse, many in our industry are turning a blind eye, hoping the problem goes away in time, without too much publicity. Why? Well, nearly everyone gets paid from fraud – the publishers, the exchanges, the data providers, and the agencies. Even the marketers,who are footing the bill, feel like they are getting value – because the success metrics they’ve set up are being  met.

But fraud hurts the ecosystem in a massive way. It means that low quality, invisible, or purely fraudulent inventory is holding down the average value of the entire marketplace – hurting high quality, engaged publishers in the process, stunting investment in quality content.

Over and over, I hear that the reason CPMs (the amount of money a marketer is willing to pay for one thousand advertising impresssions) are so low is because “there’s infinite inventory.”

Hogwash. There’s only so much time in the day, and only so many pages where actual human beings are really paying attention, and the web (including mobile) is growing at a finite pace. There are even fewer places where marketers can be assured of quality, engagement, and appropriate context. It’s time we focus on identifying them, and ridding ourselves of the true source of “infinite inventory” – fraud.