I’ve been quiet here on Searchblog these past few months, not because I’ve nothing to say, but because two major projects have consumed my time. The first, a media platform in development, is still operating mostly under the radar. I’ll have plenty to say about that, but at a later date. It’s the second where I could use your help now, a project we’re calling Mapping Data Flows. This is the research effort I’m spearheading with graduate students from Columbia’s School for International Public Affairs (SIPA) and Graduate School of Journalism. This is the project examining what I call our “Shadow Internet Constitution” driven by corporate Terms of Service.
Our project goal is simple: To visualize the Terms of Service and Data/Privacy Policies of the four largest companies in US consumer tech: Amazon, Apple, Facebook, and Google. We want this visualization to be interactive and compelling – when you approach it (it’ll be on the web), we hope it will help you really “see” what data, rights, and obligations both you and these companies have reserved. To do that, we’re busy turning unintelligible lines of text (hundreds of thousands of words, in aggregate) into code that can be queried, compared, and visualized. When I first imagined the project, I thought that wouldn’t be too difficult. I was wrong – but we’re making serious progress, and learning a lot along the way.
I’ll never forget a meal I had with a senior executive at Facebook many years ago, back when I was just starting to question the motives of the burgeoning startup’s ambition. I asked whether the company would ever support publishers across the “rest of the web” – perhaps through an advertising system competitive with Google’s AdSense. The executive’s response was startling and immediate. Everything anyone ever needs to do – including publishing – can and should be done on Facebook. The rest of the Internet was a sideshow. It’s just easier if everything is on one platform, I was told. And Facebook’s goal was to be that platform.
Those words still ring in my ears as we celebrate the 30th anniversary of the web today. And they certainly should inform our perspective as we continue to digest Facebook’s latest self-involved epiphany.
If the latest tech revelations have proven anything, it’s that the endless cycle of jaw-dropping headlines and concomitant corporate apologetics has changed exactly nothing.
Over and over, the pattern repeats. A journalist, researcher, or concerned citizen finds some appalling externality associated with one of our largest technology platforms. Representatives from the indicted company wring their hands, take down the offending content and/or de-platform the offending accounts, all the while assuring us “we actively police violations of our terms of service and are always looking to improve our service.”
This is an edited version of a series of talks I first gave in New York over the past week, outlining my work at Columbia. Many thanks to Reinvent, Pete Leyden, Cap Gemini, Columbia University, Cossette/Vision7, and the New York Times for hosting and helping me.
Every year I write predictions for the year ahead. And at the end of that year, I grade myself on how I did. I love writing this post, and thankfully you all love reading it as well. These “How I Did” posts are usually the most popular of the year, beating even the original predictions in readership and engagement.
What’s that about, anyway? Is it the spectacle of watching a guy admit he got things wrong? Cheering when I get it right? Perhaps it’s just a chance to pull back and review the year that was, all the while marveling at how much happened in twelve short months. And 2018 does not disappoint.
Here we go:
Prediction #1: Crypto/blockchain dies as a major story. Cast yourself back to late 2017 when Bitcoin was pushing $20,000 and the entire tech sector was obsessed with blockchain everything. ICOs were raising hundreds of millions of dollars, the press was hyping (or denigrating) it all, and the fools were truly rushing in. In my prediction post, I struck a more measured tone: “…there’s simply too much real-but-boring work to be done right now in the space. Does anyone remember 1994? Sure, it’s the year the Mozilla team decamped from Illinois to the Valley, but it’s not the year the Web broke out as a mainstream story. That came a few years later. 2018 is a year of hard work on the problems that have kept blockchain from becoming what most of us believe it can truly become. And that kind of work doesn’t keep the public engaged all year long.” I think I got that right. Bitcoin has crashed to earth, and those who remain in the space are deep in the real work – which I still believe to be fundamentally important to the future of not only tech, but society as well. Score: 10/10
Prediction #2: Donald Trump blows up. I don’t usually make political predictions, but by 2017, Trump was the story, bigger than politics, and bigger than tech. I wrote: “2018 is the year [Trump] goes down, and when [he] does, it will happen quickly (in terms of its inevitability) and painfully slowly (in terms of it actually resolving). This of course is a terrible thing to predict for our country, but we got ourselves into this mess, and we’ll have to get ourselves out of it. It will be the defining story of the year.” I think I also got this one right. Trump is done – nearly everyone I trust in politics agrees with that statement. I won’t recount all the reasons, but here are a few: No fewer than 17 ongoing investigations of the President and/or his organizations. A tanking stock market that has lost all faith in the President’s leadership. Nearly 40 actual indictments and several high profile guilty verdicts. A Democratic majority in the House preparing an endless barrage of subpoenas and investigations. And a Republican party finally ready to abandon its leader. Net net: Trump is toast. It’s just going to take a while for that final pat of butter. Score: 10/10
Prediction #3: Facts make a comeback. Here’s what I wrote in support of this assertion: “2018 is the year the Enlightenment makes a robust return to the national conversation. Liberals will finally figure out that it’s utterly stupid to blame the “other side” for our nation’s troubles. Several viral memes will break out throughout the year focused on a core narrative of truth and fact. The 2018 elections will prove that our public is not rotten or corrupt, but merely susceptible to the same fever dreams we’ve always been susceptible to, and the fever always breaks. A rising tide of technology-driven engagement will help drive all of this.” I’d like to claim I nailed this one, but I think the trend lines are supportive. Real journalism had a banner year, with subscriptions to high-integrity publications breaking records year on year. Most smart liberals have realized that the politics of blame is a losing game. And I was happily right about the 2018 elections, which was one of the most definitive rebukes of a sitting President in the history of our nation. As for those “viral memes” I predicted, I’m not sure how I might prove or disprove that assertion – none come to mind, but I may have missed something, given what a blur 2018 turned out to be. Alas, that “rising tide of technology-driven engagement” was a pretty useless statement. Everything these days is tech-driven…so I deserve to be dinged for that pablum. But overall? Not bad at all. Score: 7/10
Prediction #4: Tech stocks overall have a sideways year. It might be hard to give me credit for this one, given how the FANG names have tanked over the past few months, but cast your mind back to when I wrote this prediction, in late December: Tech stocks were doing nothing but going up. And where are they now? After continuing to climb for months, they’re….mostly where they started the year. Sideways. Apple started at around 170, and today is at … 156. Google started at 1048, and is now at…1037. Amazon and Netflix did better, rising double digit percentages, but plenty of other tech stocks are down significantly year on year. The tech-driven Nasdaq index started the year at around 7000, as of today, it’s down to 6600. So, some up, some down, and a whole lot of … sideways. As I wrote: “All the year-in-review stock pieces will note that tech didn’t drive the markets in the way they have over the past few years. This is because the Big Four have some troubles this coming year.” Ummm….yep, and see the next two predictions… Score: 9/10.
Prediction #6: Google/Alphabet will have a terrible first half (reputation wise), but recover after that. Well, in my original post, I predicted a #MeToo shoe dropping around Google Chairman Eric Schmidt. That didn’t happen exactly, though the whisper-ma-phone was sure running hot for the first few months of the year, and a massive sexual misconduct scandal eventually broke out later in the year. But even if I was wrong on that one point, it’s true the company had a bad first half, and for the most part, a pretty terrible year overall. In March, it had a government AI contract blow up in its face, leading to employee protests and resignations. This trend only continued throughout the year, culminating in thousands of employees walking out in protest of the company’s payouts to alleged sexual harassers. Oh, and that empty chair at Congressional hearings sure didn’t help the company’s reputation. I also predicted more EU fines: Check! A record-breaking $5 billion fine, to be exact. Further, news the company was creating a censored version of its core search engine in China also tarnished big G. But I whiffed when I mulled how the company might get its mojo back: I predicted it would consider breaking itself up and taking the parts public. That didn’t happen (as far as we know). Instead, Google CEO Sundar Pichai finally relented, showing up to endure yet another act in DC’s endless string of political carnivals. Pichai acquitted himself well enough to support my assertion that Google began to recover by year’s end. But as recoveries go, it’s a fragile one. Score: 8/10.
Prediction #7: The Duopoly falls out of favor. This was my annual prediction around the digital advertising marketplace, focused on Facebook and (again) Google. In it, I wrote: “This doesn’t mean year-on-year declines in revenue, but it does mean a falloff in year-on-year growth, and by the end of 2018, a increasingly vocal contingent of influencers inside the advertising world will speak out against the companies (they’re already speaking to me privately about it). One or two of them will publicly cut their spending and move it to other places.” This absolutely occurred. I’ve already chronicled Google’s travails in 2018, and there’s simply not enough pixels to do the same for Facebook. This New York Times piece lays out how advertisers have responded: No Morals. In the piece, and many others like it, top advertisers, including the CEO of a major agency, went on the record decrying Facebook – giving me cause for a #humblebrag, if I do say so myself. Oh, and yes, both Facebook and Google posted lower revenue growth rates year on year. Score: 10/10.
Prediction #8: Pinterest breaks out. As I wrote in my original post: “This one might prove my biggest whiff, or my biggest “nailed it.” Well, near the end of 2018, a slew of reports predicted that Pinterest is about to file for a massive IPO. As if by magic, the world woke up to Pinterest. It seems I was right – but as of yet, the IPO has not been confirmed. So…I’ll not score myself a 10 on this one, but if Pinterest does have a successful IPO early next year, I reserve the right to go back and add a couple of points. Score: 8/10.
Prediction #9: Autonomous vehicles do not become mainstream. Driverless cars have been “just around the corner” for what feels like forever. By late 2017, everyone in the business was claiming they’d breakout within a year. But that didn’t happen, regardless of the hype around the first “commercial launch” by Waymo in Phoenix a few weeks ago. I’m sorry, but a “launch” limited to 400 pre-selected and highly vetted beta ain’t mainstream – it’s not even a service in any defensible way. We’re still a long, long way off from this utopian vision. Our cities can’t even figure out what to do with electric scooters, for goodness sake. It’ll be a coon’s age before they figure out driverless cars. Score: 9/10.
Prediction #10: Business leads. I think I need to avoid these spongy predictions, because it’s super hard to prove whether or not they came true. 2018 showed us plenty of examples of business leadership along the lines of what I predicted. Here’s what I wrote: “A crucial new norm in business poised to have a breakout year is the expectation that companies take their responsibilities to all stakeholders as seriously as they take their duty to shareholders. “All stakeholders” means more than customers and employees, it means actually adding value to society beyond just their product or service. 2018 will be the year of “positive externalities” in business.” Well, I could list all the companies that pushed this movement forward. Lots of great companies did great things – Salesforce, a leader in corporate responsibility, even hired a friend of mine to be Chief Ethics Officer. Imagine if every major company empowered such a position? And a powerful Senator – Elizabeth Warren, who likely will run for the presidency in 2019 – laid out her vision for a new approach to corporate responsibility in draft legislation called the Accountable Capitalism Act. But at the end of the day, I’ve got no way to prove that 2018 was “a break out year” for “a crucial new norm in business.” I wish I did, but…I don’t. Score: 5/10.
Overall, I have to say, this was one of the most successful reviews of my predictions ever – and that’s saying something, given I’ve been doing this for more than 15 years. Nine of ten were pretty much correct, with just one being a push. That sets a high bar for my predictions for 2019…coming, I hope, in the next week or so. Until then, thanks as always for being a fellow traveler. And happy new year – may 2019 bring you and yours happiness, health, and gratitude.
Those of us fortunate enough to have lived through the birth of the web have a habit of stewing in our own nostalgia. We’ll recall some cool site from ten or more years back, then think to ourselves (or sometimes out loud on Twitter): “Well damn, things were way better back then.”
Then we shut up. After all, we’re likely out of touch, given most of us have never hung out on Twitch. But I’m seeing more and more of this kind of oldster wistfulness, what with Facebook’s current unraveling and the overall implosion of the tech-as-savior narrative in our society.
Hence the chuckle many of us had when we saw this trending piece suggesting that perhaps it was time for us to finally unhook from Facebook and – wait for it – get our own personal webpage, one we updated for any and all to peruse. You know, like a blog, only for now. I don’t know the author – the editor of the tech-site Motherboard – but it’s kind of fun to watch someone join the Old Timers Web Club in real time. Hey Facebook, get off my lawn!!!
That Golden Age
So as to not bury the lead, let me state something upfront: Of course the architecture of our current Internet is borked. It’s dumb. It’s a goddamn desert. It’s soil where seed don’t sprout. Innovation? On the web, that dog stopped hunting years ago.
And who or what’s to blame? No, no. It’s not Facebook. Facebook is merely a symptom. A convenient and easy stand in – an artifact of a larger failure of our cultural commons. Somewhere in the past decade we got something wrong, we lost our narrative – we allowed Facebook and its kin to run away with our culture.
Instead of focusing on Facebook, which is structurally borked and hurtling toward Yahoo-like irrelevance, it’s time to focus on that mistake we made, and how we might address it.
Just 10-15 years ago, things weren’t heading toward the our currently crippled version of the Internet. Back in the heady days of 2004 to 2010 – not very long ago – a riot of innovation had overtaken the technology and Internet world. We called this era “Web 2.0” – the Internet was becoming an open, distributed platform, in every meaning of the word. It was generative, it was Gates Line-compliant, and its increasingly muscular technical infrastructure promised wonder and magic and endless buckets of new. Bandwidth, responsive design, data storage, processing on demand, generously instrumented APIs; it was all coming together. Thousands of new projects and companies and ideas and hacks and services bloomed.
Sure, back then the giants were still giants – but they seemed genuinely friendly and aligned with an open, distributed philosophy. Google united the Internet, codifying (and sharing) a data structure that everyone could build upon. Amazon Web Services launched in 2006, and with the problem of storage and processing solved, tens of thousands of new services were launched in a matter of just a few years. Hell, even Facebook launched an open platform, though it quickly realized it had no business doing so. AJAX broke out, allowing for multi-state data-driven user interfaces, and just like that, the web broke out of flatland. Anyone with passable scripting skills could make interesting shit! The promise of Internet 1.0 – that open, connected, intelligence-at-the-node vision we all bought into back before any of it was really possible – by 2008 or so, that promise was damn near realized. Remember LivePlasma? Yeah, that was an amazing mashup. Too bad it’s been dormant for over a decade.
After 2010 or so, things went sideways. And then they got worse. I think in the end, our failure wasn’t that we let Facebook, Google, Apple and Amazon get too big, or too powerful. No, I think instead we failed to consider the impact of the technologies and the companies we were building. We failed to play our hand forward, we failed to realize that these nascent technologies were fragile and ungoverned and liable to be exploited by people less idealistic than we were.
Our Shadow Constitution
Our lack of consideration deliberately aided and abetted the creation of a unratified shadow Constitution for the Internet – a governance architecture built on assumptions we have accepted, but are actively ignoring. All those Terms of Service that we clicked past, the EULAs we mocked but failed to challenge, those policies have built walls around our data and how it may be used. Massive platform companies have used those walls to create impenetrable business models. Their IPO filings explain in full how the monopolization and exploitation of data were central to their success – but we bought the stock anyway.
We failed to imagine that these new companies – these Facebooks, Ubers, Amazons and Googles – might one day become exactly what they were destined to become, should we leave them ungoverned and in the thrall of unbridled capitalism. We never imagined that should they win, the vision we had of a democratic Internet would end up losing.
It’s not that, at the very start at least, that tech companies were run by evil people in any larger sense. These were smart kids, almost always male, testing the limits of adolescence in their first years after high school or college. Timing mattered most: In the mid to late oughts, with the winds of Web 2 at their back, these companies had the right ideas at the right time, with an eager nexus of opportunistic capital urging them forward.
They built extraordinary companies. But again, they built a new architecture of governance over our economy and our culture – a brutalist ecosystem that repels innovation. Not on purpose – not at first. But protected by the walls of the Internet’s newly established shadow constitution and in the thrall of a new kind of technology-fused capitalism, they certainly got good at exploiting their data-driven leverage.
So here we are, at the end of 2018, with all our darlings, the leaders not only of the tech sector, but of our entire economy, bloodied by doubt, staggering from the weight of unconsidered externalities. What comes next?
2019: The Year of Internet Policy
Whether we like it or not, Policy with a capital P is coming to the Internet world next year. Our newly emboldened Congress is scrambling to introduce multiple pieces of legislation, from an Internet Bill of Rights to a federal privacy law modeled on – shudder – the EU’s GDPR. In the past month, I’ve read draft policy papers suggesting we tax the Internet’s advertising model, that we break up Google, Facebook, and Amazon, or that we back off and just let the market “do its work.”
And that’s a good thing, to my mind – it seems we’re finally coming to terms with the power of the companies we’ve created, and we’re ready to have a national dialog about a path forward. To that end, a spot of personal news: I’ve joined the School of International and Public Affairs at Columbia University, and I’m working on a research project studying how data flows in US markets, with an emphasis on the major tech platforms. I’m also teaching a course on Internet business models and policy. In short, I’m leaning into this conversation, and you’ll likely be seeing a lot more writing on these topics here over the course of the next year or so.
Oh, and yeah, I’m also working on a new project, which remains in stealth for the time being. Yep, has to do with media and tech, but with a new focus: Our political dialog. More on that later in the year.
I know I’ve been a bit quiet this past month, but starting up new things requires a lot of work, and my writing has suffered as a result. But I’ve got quite a few pieces in the queue, starting with my annual roundup of how I did in my predictions for the year, and then of course my predictions for 2019. But I’ll spoil at least one of them now and just summarize the point of this post from the start: It’s time we figure out how to build a better Internet, and 2019 will be the year policymakers get deeply involved in this overdue and essential conversation.
A year and a half ago I reviewed Yuval Noah Harari’s Homo Deus, recommending it to the entire industry with this subhead: “No one in tech is talking about Homo Deus. We most certainly should be.”
Eighteen months later, Harari is finally having his technology industry moment. The author of a trio of increasingly disturbing books – Sapiens, for which made his name as a popular historian philosopher, the aforementioned Homo Deus, which introduced a dark strain of tech futurism to his work, and the recent 21 Lessons for the 21st Century – Harari has cemented his place in the Valley as tech’s favorite self-flagellant. So it’s only fitting that this weekend Harari was the subject of New York Times profile featuring this provocative title: Tech C.E.O.s Are in Love With Their Principal Doomsayer. The subhead continues: “The futurist philosopher Yuval Noah Harari thinks Silicon Valley is an engine of dystopian ruin. So why do the digital elite adore him so?”
Well, I’m not sure if I qualify as one of those elites, but I have a theory, one that wasn’t quite raised in the Times’ otherwise compelling profile. I’ve been a student of Harari’s work, and if there’s one clear message, it’s this: We’re running headlong into a world controlled by a tiny elite of superhumans, masters of new technologies that the “useless class” will never understand. “Homo sapiens is an obsolete algorithm,” Harari writes in Homo Deus. A new religion of Dataism will transcend our current obsession with ourselves, and we will “dissolve within the data torrent like a clump of earth within a gushing river.” In other words, we humans are f*cked, save for a few of the lucky ones who manage to transcend their fate and become masters of the machines. “Silicon Valley is creating a tiny ruling class,” the Times writes, paraphrasing Harari’s work, “and a teeming, furious “useless class.””
So here’s why I think the Valley loves Harari: We all believe we’ll be members of that tiny ruling class. It’s an indefensible, mathematically impossible belief, but as Harari reminds us in 21 Lessons, “never underestimate human stupidity.” Put another way, we are fooling ourselves, content to imagine we’ll somehow all earn a ticket into (or onto) whatever apocalypse-dodging exit plan Musk, Page or Bezos might dream up (they’re all obsessed with leaving the planet, after all). Believing that impossible fiction is certainly a lot easier than doing the quotidian work of actually fixing the problems which lay before us. Better to be one of the winners than to risk losing along with the rest of the useless class, no?
But we can’t all be winners in the future Harari lays out, and he seems to understand this fact. “If you make people start thinking far more deeply and seriously about these issues,” he said to the Times, “some of the things they will think about might not be what you want them to think about.”
Exactly, Professor. Now that I’ve departed the Valley, where I spent nearly three decades of my life, I’m starting to gain a bit of perspective on my own complicated relationship with the power structure of the place. I grew up with the (mostly) men who lead companies like Amazon, Google, Facebook and Apple, and early in the industry’s rise, it was heady to share the same stage with legends like Bezos, Jobs, or Page. But as the technology industry becomes the driving force of social rupture, I’m far more skeptical of its leaders’ abilities to, well, lead.
Witness this nearly idea-free interview with Google CEO Sundar Pichai, also in the Times, where the meticulously media-prepped executive opines on whether his industry has a role to play in society’s ills: “Every generation is worried about the new technology, and feels like this time it’s different. Our parents worried about Elvis Presley’s influence on kids. So, I’m always asking the question, “Why would it be any different this time?” Having said that, I do realize the change that’s happening now is much faster than ever before. My son still doesn’t have a phone.”
Pichai’s son not have a phone, but he is earning money mining Ethereum (really, you can’t make this shit up). I’m not sure the son of a centi-millionaire needs to earn money – but it certainly is useful to master the algorithms that will soon control nearly every aspect of human life. So – no, son, no addictive phone for you (even though my company makes them, and makes their operating systems, and makes the apps which ensure their addictive qualities).
But mining crypto currency? Absolutely!
Should Harari be proven right and humanity becomes irrelevant, I’m pretty sure Pichai’s son will have a first class ticket out of whatever mess is left behind. But the rest of us? We should probably focus on making sure that kid never needs to use it.
If you’re read my rants for long enough, you know I’m fond of programmatic advertising. I’ve called it the most important artifact in human history, replacing the Macintosh as the most significant tool ever created.
So yes, I think programmatic advertising is a big deal. As I wrote in the aforementioned post:
“I believe the very same technologies we’ve built to serve real time, data-driven advertising will soon be re-purposed across nearly every segment of our society. Programmatic adtech is the heir to the database of intentions – it’s that database turned real time and distributed far outside of search. And that’s a very, very big deal. (I just wish I had a cooler name for it than “adtech.”)”
But lately, I’m starting to wonder if perhaps adtech is failing, not for any technical reason, but because the people leveraging are complicit in what might best be called a massive failure of imagination.
I’m about to go on a rant here, so please forgive me in advance.
But honestly, who else out there is sick of being followed by ads so stupid a fourth grader could do a better job of targeting them?
Case in point is the ad above. I took this screen shot from my phone this past weekend while I was reading a New York Times article. The image – of a robe Amazon wanted me to buy – was instantly annoying, because I had in fact purchased a robe on Amazon several days before. Why on earth was Amazon retargeting me for a product I just bought?!
But wait, it gets worse! As I perused the next Times article, this ad shows up:
You might think this ad makes more sense. If the dude buys a robe, makes sense to try to sell him a new pair of slippers, no? Well, sure, but only if that same dude didn’t buy a new pair of slippers two weeks ago. Which, in fact, I did just do.
So, yeah, this ad sucks as well. Not only is it not useful or relevant, it’s downright annoying. The vast machinery of adtech has correctly identified me as a robe-and-slippers-buying customer. But it’s failed to realize *I’ve already bought the damn things.*
Is it possible that adtech is this stupid? This poorly instrumented? I mean, are programmatic buyers simply tagging visitors who land on ecommerce pages (male robe intender?) without caring about whether those visitors actually bought anything?
Are the human beings responsible for setting the dials of programmatic just this lazy?
I’ve been a critical observer of adtech over the past ten or so years, and one consistent takeaway is this: If there’s a way for a buyer to cut corners, declare an easy win, and keep doing things they way the’ve always been done, well, they most certainly will.
But why does it have to be this way? Digging into the examples above yields an extremely frustrating set of facts. Consider the data the adtech infrastructure either got *right* about me as a customer, or could have gotten right:
I am a frequent ecommerce customer, usually buying on Amazon
I recently purchased both a robe and some slippers
I am reading on the New York Times site as a logged on (IE data rich) customer of the Times‘ offerings
These are just the obvious data points. My mobile ID and cookies, all of which are available to programmatic buyers, certainly indicate a high household income, a propensity to click on certain kinds of ads, a rich web browsing history reflecting a thickly veined lodestar of interest data, among countless other possible inputs.
Imagine if a programmatic campaign actually paid attention to all this rich data? Start with the fact I just purchased a robe and slippers. What are products related to those two that Amazon might show me? Well, according to its own “people who bought this item also bought” algorithms, folks who bought men’s robes also bought robes for the women in their life. Now there’s a cool recommendation! I might have clicked on an ad that showed a cool robe for my wife. But no, I’m shown an ad for a product I already have.
I’ve got a few calls in to verify my hunch, but I suspect the ugly truth is pure laziness on the part of the folks responsible for buying ads. Consider: The average cost for a thousand views (CPM) of a targeted programmatic advertisement hovers between ten cents (yes, ten pennies) to $2. With costs that low, the advertising community can afford to waste ad inventory.
Let’s apply that reality to our robe example. Let’s say the robe costs $60, and yields a $20 profit for our e-commerce advertiser, not including marketing costs. That means that same advertiser is can spend upwards of $19.99 per unit on advertising (more, if a robe purchaser turns out to be a “big basket” e-commerce spender). So what does our advertiser do? Well, they set a retargeting campaign aimed anyone who ever visited our erstwhile robe’s page. With CPMs averaging around a buck, that robe’s going to follow nearly 20,000 folks around the internet, hoping that just one of them converts.
Put another way, programmatic advertising is a pure numbers game, and as long as the numbers show one penny of profit, no one is motivated to make the system any better. I’ve encountered many similar examples of ad buyers ignoring high-quality data signals, preferring instead to “waste reach” because, well, it’s just easier to set up campaigns on one or two factors. Inventory is cheap. Why not?
This is problematic. What’s the point of having all that rich (and hard won) targeting data if buyers won’t use it, and consumers don’t benefit from it? An ecosystem that fails to encourage innovation will stagnate and lose share to walled gardens like Facebook, Google, and others. If the ads suck on the open web (and they do), then consumers will either install ad blockers (and they are), or abandon the open web altogether (and they are).
In my last post I imagined a world in which large data-driven platforms like Amazon, Google, Spotify, and Uber are compelled to share machine-readable copies of data to their users. There are literally scores, if not hundreds of wrinkles to iron out around how such a system would work, and in a future post I hope to dig into some of those questions. But for now, come with me on a journey into the future, where the wrinkles have been ironed out, and a new marketplace of personally-driven information is flourishing. We’ll return to one of the primary examples I sketched out in the aforementioned post: A battle for the allegiance – and pocketbook – of one online shopper, in this case, my wife Michelle.
It’s a crisp winter mid morning in Manhattan when the doorbell rings. Michelle looks up from her laptop, wondering who it might be. She’s not expecting any deliveries from Amazon, usually the source of such interruptions. She glances at her phone, and the Ring app (an Amazon service, naturally) shows a well dressed, smiling young woman at the door. She’s holding what looks like an elegantly wrapped gift in her hands. Now that’s unusual! Michelle checks the date – no anniversaries, no birthdays, no special occasions – so what gives?
Michelle opens the door and is greeted by a woman who introduces herself as Sheila. She tells Michelle she’s been sent over by Walmart. Walmart? Michelle’s never set foot in a Walmart store, and has a less than charitable view of the company overall. Why on earth would Walmart be sending her a special delivery gift box?
Sheila is used to exactly this kind of response – she’s been trained to expect it, and to manage the conversation that ensues. Sheila is a college-educated Walmart management associate, and delivering these gift boxes is a mandatory part of her company training. In fact, Sheila’s future career trajectory is based, in part, on her success at converting Michelle into becoming a Walmart customer, and she’s learned from her colleagues back at corporate that the best way to succeed is to be direct and open while engaging with a top-level prospect.
“Michelle, I know this seems a bit strange, but Walmart has identified you as a premier ecommerce customer – I’m guessing you probably have at least three or four packages a week delivered here?”
“More like three or four a day,” Michelle answers, warming to Sheila’s implied status as a premium customer.
“Yes, it’s amazing how it’s become a daily habit,” Sheila answers. “And as you probably know, Walmart has an online service, but truth be told, we never seem to get the business of folks like you. I’m here to see if we might change that.”
Michelle becomes suspicious. It doesn’t make sense to her – sending over a manager bearing gifts? Such tactics don’t scale – and feel like an intrusion to boot.
Sensing this, Sheila continues. “Look, I’m not here to sell you anything. I’ve got this special gift for you from Doug McMillon, the CEO of Walmart. You’ve been selected to be part of a new program we’re testing – we call it Walton’s Circle. It’s named after Sam Walton, our founder, who was pretty fond of the personal touch. In any case, the gift is yours to keep. There’s some pretty cool stuff in there, I have to say, including La Mer skin cream and some Neuhaus chocolate that’s to die for.”
Michelle smiles. Strange how the world’s biggest retailer, a place she’s never shopped, seems to know her brand preferences for skin care and chocolate. Despite herself, she relaxes a bit.
“Also inside,” Sheila continues, “is an invitation. It’s entirely up to you if you want to accept it, but let me explain?”
“Sure,” Michelle answers.
“Great. Have you heard of the Token Act?”
Michelle frowns. She read about this new piece of legislation, something to do with personal data and the right to exchange it for value across the internet. In the run up to its passage, her husband wouldn’t shut up about how revolutionary it was going to be, but so far nothing important in her life had changed.
“Yes, I’ve heard of it,” Michelle answers, “but it all seems pretty abstract.”
“Yeah, I hear that all the time,” Sheila responds. “But that’s where our invitation comes in. Inside the box is an envelope with a code and a website. I imagine you use Amazon…” Sheila glances toward an empty brown box in the hallway with Amazon’s universal smiling logo. Michelle laughs. “Of course you do! I was a huge Amazon customer for years. And that’s what our invitation is about – it’s an invitation to see what might happen if you became a Walmart customer instead. If you go to our site and enter your code, a program will automatically download your Amazon purchase history and run it through Walmart’s historical inventory. Within seconds, you’ll be given a report detailing what you would have saved had you purchased exactly the same products, at the same time, from us instead of Jeff Bezos.”
“Huh,” Michelle responds. “Sounds cool but…that’s my information on Amazon, no? I don’t want you to have that, do I?”
“Of course not,” Sheila says knowingly. “All of your information is protected by LiveRamp Identity, and is never stored or even processed on our servers. You maintain complete control over the process, and can revoke it at any time.”
Michelle had heard of LiveRamp Identity, it was a third-party guarantor of information safety she’d used for a recent mortgage application. She also came across it when co-signing for a car loan for her college-aged daughter.
“When you put that code into our site, a token is generated that gives us permission to compare our data to yours, and a report is generated,” Sheila explained. “The report is yours to keep and do with what you want. In fact, the report becomes a token in and of itself, and you can submit that token to third party services like TokenTrust, which will audit our work and tell you if our results can be trusted.”
TokenTrust was another service Michelle had heard of, her husband had raved about it as one of the fastest growing new entrants in the tech industry. The company had recently been featured on 60 Minutes – it played a significant role in a story about Google’s search results, if she recalled correctly. Docusign had purchased the company for several billion just last year. In any case, Michelle’s suspicions were defused – may as well check this out. I mean, why would Walmart risk its reputation stealing her Amazon data? It was worth at least seeing that report.
Sheila sensed the opening. “The reports are pretty amazing,” she says. “I’ve had clients who’ve discovered they could have saved thousands of dollars a year. And here’s the best part: If, after reviewing and validating the report, you switch to Walmart, we’ll credit your account with those savings – in essence, we’ll retroactively deliver you the savings you would have had all along.”
“Wow. That almost sounds too good to be true!” Michelle says. “But… OK, thanks. I’ll check it out. Thanks for coming by.”
“Absolutely,” Sheila responds. “And here’s my card – that’s my cell, and my email. Let me know if you have any questions.”
Michelle heads back inside and places the gift box on the table next to her laptop. Before opening the box, she wants to be sure this thing is for real. She Googles “Walmart Walton Circle Savings Token” – and the first link is to a Business Insider article: “These Lucky Few Amazon Customers Are Paid Thousands to Switch – By Walmart.” So Sheila wasn’t lying – this program is for real!
Michelle tugs on the satin ribbon surrounding her gift box and raises its sturdy lid. Nestled on straw inside are two jars of La Mer, several samples of Neuhaus chocolates, two of her favorite bath salts, and various high end household items. The inside lid of the box proclaims “Welcome to Walton’s Circle!” in elegant script. At the center of the box is an creamy envelope engraved with her name. Michelle opens it, and just as Sheila mentioned, a URL and code is included, along with simple instructions.
What the hell, may as well see what comes of it. Turning to her laptop, Michelle heads to Walmart.com – for the first time in her life – and enters her code. Almost instantaneously a dialog pops up, informing her that her report is ready. Would she like to review it?
Why not?! Michelle clicks “Yes” and up comes a side-by-side comparison of her entire Amazon purchase history. She notices that during the early years – roughly until 2006 – there’s not much on the Walmart side of the report. But after that the match rates start to climb, and for the past five or so years, the report shows that 98 percent of the stuff she’s bought at Amazon was also available on Walmart.com. Each purchase has a link, and she tries out one – a chaise lounge she purchased in 2014 (gotta love Prime shipping!). Turns out Walmart didn’t have that exact match, but the report shows several similar alternatives, any of which would have worked. Cool.
Michelle’s eye is drawn to the bottom of the report, to a large sum in red that shows the difference in price between her Amazon purchases and their Walmart doppelgangers.
Holy….cow. Michelle can’t believe it. Is this for real? Anticipating the question, Walmart’s report software pops up a dialog. “Would you like to validate your token’s report using TokenTrust? We’ll pay all fees.” Michelle clicks yes, and a TokenTrust site appears. The site shows a “working” icon for several seconds, then returns a simple message: “TokenTrust has reviewed Walmarts claims and your Amazon token, and validates the accuracy of this report.”
Michelle is sold. Next to the $2700 figure at the bottom of her report is one line of text, and a “Go” link. “Would you like to become a founding member of the Walton Circle? We’ll take care of all your transition needs, and Sheila, who’ve you already met, will be named as your personal shopping concierge.”
Michelle hovers momentarily over “Go.” What the hell, she thinks. I can always switch back. And with one click, Michelle does something she never thought she would: She becomes a Walmart customer.
Satisfied, she turns her eyes back to her work. Several new emails have collected in her inbox. One is from Doug McMillon, welcoming her to Walton’s Circle. As she hovers over it, mail refreshes, and a new message piles on top of McMillon’s.
Holy shit. Did Jeff Bezos really just email me?!
Is such a scenario even possible? Well, that question remains unexplored, at least for now. As I wrote in my last post, I’m not certain Amazon’s terms of service would allow for such an information exchange, though it’s currently possible to download exactly the information Walmart would need to stand up such a service. (I’ve done it, it takes a bit of poking around, but it’s very cool to see.) The real question is this: Would Walmart spend the thousands of dollars required to make this kind of customer acquisition possible?
I don’t see why not. A high end e-commerce customer spends more than ten thousand dollars a year online. Over a lifetime, this customer is worth thousands of dollars in profit for a well-run commerce site like Walmart. The most difficult and expensive problem for any brand is switching costs – it’s at the core of the most sophisticated marketing efforts in the world – Ford spends hundreds of millions each year trying to convince customers to switch from GM, Verizon spends equal amounts in an effort to pull customers from AT&T. Over the past five years, Walmart has watched Amazon run away with its customers online, even as it has spent billions building a competitive commerce offering. What Walmart needs are “point to” customers – the kind of people who not only become profitable lifelong buyers, but who will tell hundreds of friends, family members and colleagues about their gift box experience.
But to get there, Walmart needs that Amazon token. Wouldn’t it be cool if such a thing actually existed?
Yesterday, I lost it over a hangnail and a two-dollar bottle of hydrogen peroxide.
You know when a hangnail gets angry, and a tiny red ball of pain settles in for a party on the side of your finger? Well, yeah. That was me last night. My usual solution is to stick said finger into a bottle of peroxide for a good long soak. But we were out of the stuff, so, as has become my habit, I turned to Amazon. And that’s when things not only got weird, they got manipulative. Sure, I’ve been ambiently aware of Amazon’s algorithmic pricing and merchandising practices, but last night, the raw power of the company’s control over my routine purchases was on full display.
There’s literally no company in the world with better data about online purchasing than Amazon. So studying how and where it lures a shopper through a purchase process is a worthy exercise. This particular one left a terrible taste in my mouth – one I don’t think I’ll ever shake.
First the detail. Take a look at my search results for “Hydrogen Peroxide” on Amazon. I’ve annotated them with red text and arrows:
As you can see, the most eye catching suggestions – the four featured panels with large images – are all Amazon brands. Big red flag. But Amazon knows sophisticated shoppers like me are suspicious of those in house suggestions, so it’s included a similar product in the space below its own brands (we’ll get to that in a minute).
Above the featured items are ads: sponsored listings that are not Amazon brands, which means the advertiser (a small player named “Blubonic Industries”) is paying Amazon to get ahead of the company’s own promotional power. Either way, Amazon makes money. Second red flag.
By now, I’ve decided I’m not interested in either the sponsored brands at the top, or Amazon’s four featured brands, because, well, I don’t like to be so baldly steered into buying Amazon’s stuff. Then again, before I move down to the results below, I do notice something rather amazing – Amazon’s familiar brown bottle of peroxide is really, really cheap – as in, $1.29 cheap. There’s even a helpful per oz. calculation next to the price, screaming: this shit is eight pennies an ouncecheap!
Well, I’m almost sold, but because I hate to be directed into purchases, I’m still going to consider that similar brown bottle below, the one with the red label. Amazon knows this, of course. It’s merchandising 101 – make sure you give the consumer choices, but also, make sure the most profitable choice is presented in such a way as to win the day.
So my eye moves down the page to check out the second bottle. It’s from Swan, a brand I’ve vaguely heard of. Then I check its price.
Nine dollars and sixty nine cents.
Which would you buy? After all, this is a staple, a basic, a chemical compound. And you trust Amazon to get shit right, don’t you? I mean, a buck and change – nearly nine times cheaper? What a deal!
So…my eyes revert to Amazon’s blue labeled bottle. It wouldn’t have a four-star plus review if it burned your skin, right? And that’s when I notice the tiny icon next to it, which looks like this:
What’s this? Is this yet another annoying subscription service? Ever since we moved to New York, my wife and I have tried to figure out Amazon’s subscription services (Fresh? Pantry? Prime Now? Whole Foods Delivery? Who knows?!). I’m already deeply suspicious of any attempt by Amazon to lure me into paying them monthly for a service that I don’t understand.
But…a buck twenty nine! So I click on the bottle, and the landing page is super clean, and there’s no obvious Prime Pantry mention. Plus, it turns out, that bottle from Amazon is the Whole Foods generic brand, which for whatever reason seems a bit better than a generic Amazon brand. Did I just get lucky? Maybe I can just get some super cheap chemicals delivered in a day to my door, and my annoying hangnail will be a thing of the past soon enough….Right?
Here’s the landing page:
Looks great, the price is amazing, but…Uh oh. I can’t get this bottle of peroxide until Sunday. By then, I’ve likely lost my finger to a flesh eating bacteria. As I feared, this bottle is nothing more than a baited fish hook for one of Amazon’s subscription offers – which I find out, will cost somewhere between five and thirteen bucks a month. I’ve signed up for Prime Pantry by mistake in the past, and it wasn’t a smooth or enjoyable experience. No thanks. I click back to the original search results. Seems to me Amazon is gaming the shipping deals.
Well of course it is. I’m no longer a happy Amazon customer at this point. Now I’m annoyed.
But what’s this? If I scroll down below the $9.69 bottle, there’s another choice, also from Swan, and, it seems, exactly the same, if one is to judge just by the image (and we do judge just from the images, let’s just admit it). This one costs almost half as much as the one above it. What’s going on?! Here’s an annotated screen shot:
As you can see, there’s a lot going on. I’ve narrowed my choice down to two non-Amazon brands. They look nearly identical. The most significant difference, at least in terms of the information provided to me by Amazon, is the price – the top bottle is nearly twice as expensive as the bottom one. But the top bottle has a major benefit: I can get it nearly immediately! The bottom one makes me wait a day. Is the wait worth four or five bucks? Hmm.
Also confounding: The bottom bottle has its price broken out on a per ounce basis – 32 cents, exactly four times more than the 8 cents-an-ounce bottle I just looked at from Amazon’s Prime Pantry. Ouch! Now I’m really annoyed, and confused. My eyes dart back up to the $9.69 bottle. As I’ve shown with the empty red circle, there’s….no per-ounce breakdown shown by Amazon. It does tell me that this particular bottle is 32 ounces, whereas the bottom one is 16 ounces.
But why not do the math for me? A quick calculation shows that the top bottle comes out to about 30 cents an ounce – two cents less than the bottom bottle. Why not show that fact?
This, folks, this is algorithmic merchandising at its finest.
Amazon knows exactly how many clicks it’s going to take for me to reach shopping fatigue. Not “on average for all shoppers,” or even “on average for each shopper who’s ever considered a bottle of hydrogen peroxide.” Amazon knows all of that, of course, but it also knows exactly how long it takes ME to get fatigued, to enter what I like to call “fuck it” mode. As in, “fuck it, I’m tired of this bullshit, I want to get back to the rest of my life. I’m going to buy one of these bottles.”
And because there’s no per-ounce breakdown of the 32-ounce bottle, and because that makes me suspicious of it, and because hell, who ever needs 32 ounces of hydrogen peroxide anyway, well, I’m just going to buy the $5 one.
Ca-ching! Amazon just made a nearly seven percent markup on my purchase. It took five clicks, 15 seconds, and a vast architecture of data and algorithmic mastery to make that profit. Each and every time we purchase something on Amazon, that machinery is engaged in the background, guiding us through choices which insure the company remains the trillion dollar behemoth we know and…
Do you love Amazon anymore? For that matter, do you love Facebook, Google, or Twitter? Interactions like the one I’ve detailed above are starting to chip away at that presumption. Personally, I’ve gone from cheerleader to skeptic over the past few years, and I’m broken out into full-blown critic over the last twelve months. I no longer trust Amazon to have my best interests at heart. I’ve lost any trust that Facebook or Twitter can deliver me a public square representative of my democracy. I’ve given up on Google delivering me search results that are truly “organic.” And YouTube? Point solution, at best. I can’t possibly trust the autoplay feature to do much more than waste my time.
What’s happened to our beloved tech icons, and what are the implications of this lost trust? In future posts, I plan on thinking out loud on that topic. I hope you’ll join me. In the meantime, I think I’ll stroll down to CVS and buy myself another bottle of hydrogen peroxide. By the time Amazon’s comes, I’m sure my hangnail will be a distant memory. But that taste in my mouth? That’s going to remain.
Update: Many readers have pointed out that I missed the fact that the top package of peroxide was, in fact, a two-pack. True that, and it would have changed my on-the-fly calculation around which to buy, given the per ounce comparison. However, it would not change the fact that the act of not adding the per ounce calculation directly on the page somehow discolored that choice.
Also, a rather rich post note: The bottle I did buy never came. It was “lost” – and Amazon offered me a refund. Sometimes it pays to just hit CVS.