Tag: search

Where’s All the AI Magic?

“Hey Google, how are the Giants doing this year?” 

I was standing at my bathroom sink, finishing up my daily ablutions, when a random thought popped into my head. It’s been a minute since I checked in on my favorite baseball team – ever since I moved East, they’ve been in a slump. Maybe they’re pulling back into contention this year? I still have a Google Home plugged in nearby. I’m intimately familiar with the Home’s limitations, but I asked anyway. Perhaps I was hoping one of my annual predictions (voice interfaces for the home) would magically come true.

Read More
1 Comment on Where’s All the AI Magic?

SIGN UP FOR THE NEWSLETTER

Stay up to date on the latest from BattelleMedia.com

How Search Drove Generative AI – A Passage from “The Search”

It’s been fun to go back to Berkeley, where I first taught Journalism more than 20 years ago. I’m leading a seminar on how technology impacts journalism, with a particular focus on AI. The class asks students to read a bit of history – it’s hard to understand where we are if we don’t know how we got here. Search is a big part of that history, so I included a chapter of my first book – The Search – as a reading assignment.

As I prepared for class last week, I dug through my archives and unearthed The Search’s original manuscript. In the first chapter, “The Database of Intentions,” I opine on how search might lead to the development of AI that passes the Turing Test. Written 22 years ago, the passage anticipates the rise of generative AI. I start by drawing a distinction between data that is on our personal machines and data held in the cloud by large technology companies like Google. Then I think out loud a bit about where that all data might take us. Even though the writing is two decades old, it prompts some interesting questions about the moment in which we find ourselves.

—

When our data is on our desktop, we assume that it is ours. It’s my address book that lives in Entourage, my email attachments, and my hard drive inside my Powerbook. When I am looking for a file or a particular email message on my local files (when I am searching my local disk), I presume that my mouse-and-click actions – that of searching, finding, manipulating data – are not being watched, recorded, and analyzed by a third party for any reason, be it benign or malicious. (In certain workplaces, this is certainly no longer the case, but we’ll set that aside for now.)

But when the locus of computing moves to the web, as it clearly is for second generation applications like social networking, search, e-commerce, and the like, the law is far fuzzier. What of the data that is stored and created through interactions with those applications? Who owns that data? What rights to it do we have? The truth is, at this point, we just don’t know.

As we move our data to the servers at amazon.com, hotmail.com, yahoo.com, and gmail.com, we are making an implicit bargain, one that the public at large is either entirely content with, or, more likely, one that most have not taken much to heart.

That bargain is this: We trust you to not do evil things with our information. We trust you will keep it secure, free from unlawful government or private search and seizure, and under my control at all times. We understand you might use my data in aggregate to provide us better and more useful services, but we trust that you will not identify me personally through my data, nor use my personal data in a manner that would violate my own sense of privacy and freedom.

That’s a pretty large helping of trust we’re asking companies to ladle onto their corporate plate. And I’m not sure either we – or they – are entirely sure what to do with the implicit and explicit implications of such a transfer. Just thinking about these implications makes a reasonable person’s head hurt.

But imagine the disorientation you might feel if search becomes self aware – capable of watching you as you interact with it?

Search As Artificial Intelligence?

“I would like to see the search engines become like the computers in Star Trek,” Google employee number one, Craig Silverstein, often quips. “You talk to them and they understand what you’re asking.”

Silverstein, a soft-spoken paragon of Google’s geek culture, is hardly kidding. The idea that search will one day morph into a human like form pervades nearly all discussion of the application’s future. Asked at a conference how he’d best describe his search service, Ask Jeeves executive Paul Gardi replied: “(The android character) Data from Star Trek. We know everything you might need.”

But how might we get there? For search to cross into intelligence, it must understand a request – the way you, as a reader, understand this sentence (one hopes). “My problem is not finding something,” said Danny Hillis, a MacArthur-certified genius and computer scientist who now runs a consulting business. “My problem is understanding something.” That, he continued, can only happen if search engines understand what a person is really looking for, and then guide him or her toward understanding that thing, much as experts do when mentoring a student. Search, he continued, “is an obvious place for intelligence to happen, and it is starting to happen.”

So Hillis argues that the future of search will be more about understanding, rather than simply finding. But can a machine ever understand what you are looking for? Answering that question raises what is perhaps computing’s holiest of grails: passing the Turing test.

The Turing test, developed by British mathematician Alan Turing in a seminal 1950 article, lays out a model to prove whether or not a machine can be considered intelligent. While the test and its prescripts are subject to intense academic debate, the general idea is this: an interrogator is blindly connected to two entities, one a machine, the other a person. The questioner has no idea which is which. His task is to determine, through questioning both, which is man, and which is machine. If a machine manages to “fool” the questioner into believing it is human, it has passed the Turing test and can be considered intelligent.

Turing predicted that by the year 2000 computers would be smart enough to have a serious go at passing the Turing test. He was right about the “serious go” part, but so far, the prize has eluded the best and brightest in the field. In 1990 a wealthy oddball, Hugh Loebner, offered $100,000 to the first computer to pass the test. Every year, AI companies line up to win the honor. Every year, the money remains uncollected.

That may well be because, as with so many things, people are framing the problem in the wrong way. So far, contestants have focused on building singular “robots” which have millions of potential answer sequences coded in, so that for any particular question a plausible answer might be given. Perhaps the most famous of these efforts is CYC (pronounced “psych”), the life’s work of AI pioneer Doug Lenant. CYC attempts to conquer AI’s “brittleness problem” by coding in hundreds of thousands of “common sense” rules – mountains go up, then down, valleys are between hills or mountains, etc. – and then build a robust model based on those simple rules. Not surprisingly, a CYC alumnus, Srinija Srinivasan, was one of Yahoo’s first employees, and has run Yahoo’s directory- based search product from nearly day one.

But brute force by one organization has failed so far, and most likely will fail in the future. No, search will more likely become intelligent via the clever application of algorithms which harness and leverage the intelligence already extant on the web – the millions and millions of daily transactions, utterances, behaviors, and links that form the web’s foundation – the Database of Intentions. After all, that’s how Google got its start, and if any company can claim to have created an “intelligent” search engine, it’s Google.

“The goal of Google and other search companies is to provide people with information and make it useful to them,” Silverstein told me. “The open question is whether human-level understanding is necessary to fulfill that goal. I would argue that it is.”

What does the world want? Build a company that answers that question in all its shades of meaning, and you’ve unlocked the most intractable riddle of marketing, business, and arguably of human culture itself. And for the past few years, Google seems to have built just that company.

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

Leave a comment on How Search Drove Generative AI – A Passage from “The Search”

AI and Ads: Here We Go!

Google launched as a free public beta in the Fall of 1998. It was a revelation – a 10X improvement on Internet navigation and research. But from its launch forward, Google’s founders were hounded with questions as to how their company planned on actually making money. John Doerr, one of Google’s earliest backers, famously answered that question by citing Google’s extraordinary growth: With all that traffic, he said, we’ll figure it out.

Google’s founders were famously suspicious of advertising – in their white paper explaining Google’s PageRank technology, Larry Page and Sergey Brin argued that advertising-funded search engines would be “inherently biased towards the advertisers and away from the needs of consumers.”

Two years later, Google launched AdWords, which became a multi-hundred-billion dollar advertising business over the following two decades.

We all knew that OpenAI was going to follow Google’s path into advertising, and late last week the company made it official. There is *a lot* to unpack around this announcement, but it’ll have to wait: I’m starting a new gig this week, teaching a seminar on the impact of technology at the UC Berkeley Graduate School of Journalism. I’ll be spending a fair bit of time in my former home of Northern California, and once I get settled, I’ll return to a more regular posting schedule here. In the meantime, if you find yourself in NorCal and you’d like to reconnect, let me know!

 

1 Comment on AI and Ads: Here We Go!

Is OpenAI Today’s Netscape? Or Is It AOL?

As is his want, last week Fred Wilson wrote a provocative post I’ve been thinking about for the past few days. Titled “Netscape and Microsoft Redux?“, Fred notes the parallels between the browser wars of the late 1990s and the present-day battle for dominance in the consumer AI market. And he asks a prescient question: What new, world-defining product might we be missing by focusing on AI chatbots?

In the early days of the Web, everyone thought the most important new product to emerge from the Internet was the browser. Netscape, a startup with just a few months of operating history, defined the market for those browsers in 1994, then dominated it for several years thereafter. But by the late 1990s, the lumbering incumbent Microsoft had stolen Netscape’s lead by leveraging distribution and pricing advantages inherent to its massive Windows monopoly.

But here’s the rub, as Fred points out: “Ironically, that battle for Internet dominance missed that the most important piece of software was the search engine, not the browser. And so the winner ended up being an entirely different company – Google.”

Fred notes that today’s version of the browser wars is playing out in chatbots, with OpenAI playing the role of the upstart (Netscape), and Google the incumbent (Microsoft). Sure, OpenAI has the lead today, but Google has woken up, and is using its dominance in search, infrastructure, and browsing to take share from its upstart competitor.

But if that metaphor holds, Fred wonders, are we once again missing “the most important new piece of software,” just as we did around search in the late ’90s? And if so, what is it?

I keep turning these questions over in my head, and it feels like an answer is tantalizingly close, but still out of reach. So I’m doing what I always do when faced with these kinds of puzzles: I’m thinking out loud through writing.

Let’s start by identifying what made search the breakout winner of the early Internet era. By the late 1990s, the dominant tool for accessing  the Internet was the browser. Those browsers let you surf an endless wave of sites, but they didn’t help you find the sites mattered to you.

The early Web had a navigation problem, and its first solution was the directory – Yahoo!, the first directory built for every day consumers, became the most visited site on the Web. But directories were soon swamped by waves of new sites coming online each month.

Google solved the Web’s navigation problem by continually crawling and indexing every site on the Internet, then delivering just the right answer based on a novel signal that had been overlooked by everyone: The link.

Google PageRank algorithm refined links – human-built connections between pages on the web –  into the most successful business in the history of the Internet, changing consumer behavior in the process. By the early 2000s, consumers began using Google as their first stop on the Web. Google became a verb, search became a habit, and portals like Yahoo languished.

Which leads me to wonder: If consumer AI really is in its “early 1990s” phase, what’s the equivalent of the navigation problem we all encountered prior to the emergence of Google search? And is there a novel signal – a new class of data emerging from our use of AI that corresponds to the link?

Exploration of these questions is complicated by how the Web works today. Twenty five years ago the Web was essentially a massive public commons. The vast majority of sites were open and free, and very few sites opted out of Google’s approach to crawling the Web. When people made new pages and links – essentially new data for Google to ingest – Google simply indexed that data, then used PageRank to figure out a way to make sense of it all. It didn’t have to ask permission – people gave it freely as a matter of course.*

Today’s Internet is markedly different. Most sites – in particular large platforms like Amazon, OpenAI, TikTok or Meta’s Instagram – operate as walled gardens that refuse to share data. And while people are still building websites, the majority of valuable data are locked behind walls of personalization and corporate terms of service. That means there’s no equivalent to the link in today’s AI-driven Internet – no novel public resource waiting to be exploited for a breakthrough service like search back in the day.

While we’ve not found an obvious analog to the link, perhaps I’ve been putting the horse before the cart. Let’s think for a spell about the problem. By the late 1990s, the problem of navigation on the Internet was widely understood. Is there a similar problem now for users of AI chat services?

The first thing that comes to mind is this: You can’t effectuate anything with services like ChatGPT or Gemini. You can’t ask them to take action on your behalf. Sure, you can ask them to write you an essay, act as a friend or therapist (or more), summarize a white paper, etc., etc., but as soon as you want to do something, chatbots grind to a halt.  Consumer AI has a “getting shit done” problem – they exist in rarified silos, incapable of anything that requires engagement beyond the confines of their chatbot interfaces.

Certainly the tech industry knows about this problem – and it has devised a solution: Agents. The next wave of AI innovation centers on “the agentic web,” with personalized agents that will do our bidding in every imaginable way. Every major AI company has announced agentic products, but unfortunately, they don’t work, because the ecosystem in which they operate is hostile to their success. Want an agent that compares prices across commerce sites like Amazon or Walmart, then makes a purchase? Sorry, ‘user agents’ are blocked by Amazon’s terms of service. In fact, nearly all commercial Internet services block ‘non human’ user agents from engaging with their sites. It’s not hard to understand why: Non human agents are, well, not human, and most sites depend on advertising revenue that’s targeted to humans, after all. Plus, user agents threaten to undermine the information asymmetry that underpins most of capitalism these days – once you’ve tasted the profits driven by dynamic pricing, it’s hard to go back.

What we have here is an architectural problem: The Internet as currently built simply cannot support an agentic future, no matter how many well-manicured hands might wave at it. Realizing that future will require a fundamental redesign of the Internet – a process as radical as the invention of the Web itself.

This leads me to an unexpected conclusion when it comes to pondering Fred’s timeline comparing the late 1990s to now. Perhaps, in fact, he’s off by a decade. Could it be that consumer AI is comparable not to the early Web, but rather to the era of nearly forgotten online services that preceded the early Web in the late 1980s? When the history of consumer AI is written, might it treat OpenAI, Claude and Gemini as the equivalent of pre-Web services like CompuServe, AOL, and MSN? These fascinating but frustrating services attempted to build online worlds, but they were built on brittle architectures that couldn’t connect easily and reliably to the “rest of the world.” The original Web offered a better model. Maybe it’s time to abandon the Web as we now know it, and dream up something entirely new once again.

—

*Plus, there was an explicit deal that quickly developed: Letting Google crawl your site meant inclusion in its index, resulting in visitors and potential business opportunities via advertising and/or commerce. 

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

3 Comments on Is OpenAI Today’s Netscape? Or Is It AOL?

Google (and Apple) Get a Slap on The Wrist

Back in the day when I was covering Google on a daily basis, I’d have spent hours poring over yesterday’s news that the judge in Google’s landmark antitrust case essentially blinked. But twenty-odd years of experience leaves me with very little to say about how Google’s first anti-trust case has been resolved, other than this: It’s a nothing burger, with a side of same-as-it-ever-was.

Over the course of nearly four years since the government brought its case, a lot has changed:

  • The United States has veered away from liberal democracy toward illiberal autocracy, and the current administration is no longer interested in grand antitrust remedies that serve the public. Today, everything is seen through the lens of whether a given action or decision furthers the President’s power. Preserving the status quo gives him leverage over powerful actors – he can continue to threaten and bully, ensuring fealty and tribute. In this administration, as in Moscow, Pyongyang and Beijing, no one is allowed to have more power than the Dear Leader.
  • As I laid out in my predictions nine months ago, the tech industry is now the most powerful force in politics outside the President, and its two most muscular companies – Apple and Google – did not want their duopoly upended. We’ll likely never know what soft-power backroom deals were cut to avoid what nearly every legal scholar felt was justified action by the government, but to think those dynamics didn’t impact this decision is to ignore the reality of my first point above.
  • OpenAI’s existence became a convenient foil. The emergence of generative AI has given Google (and the judge in this case) the ability to argue that the DOJ was fighting yesterday’s war. Sure, Google might have been a search monopolist, but look – OpenAI is proof that the market is always smarter than government regulators! Never mind the fact that search literally built the foundation for generative AI, or that generative AI is the natural evolution of search – a product that Google will continue to dominate now that government remedies have been rendered toothless.

Google – and its $20 billion partner Apple – are likely doing cartwheels today. Wall Street certainly is.

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

 

1 Comment on Google (and Apple) Get a Slap on The Wrist

Is AI Killing Search Marketing?

(This piece is cross-posted from Signal360, where it first appeared)

OK, Google, What’s The Business Model?

“How did you go bankrupt?” Bill asked.
“Two ways,” Mike said. “Gradually and then suddenly.”
                   -Ernest Hemingway, The Sun Also Rises

If you work in marketing, Hemingway’s famous passage about bankruptcy will likely resonate. The launch of ChatGPT in 2022 prompted dire predictions that search’s role as a reliable driver of traffic and sales was coming to a close. For the past three years, however, those predictions seemed overwrought. Despite a slight dip in January, which proved temporary, Google has maintained a steady grip on 90 percent of search activity, and continues to direct a firehose of leads and commerce across the web.

But rather suddenly — in the past two months, in fact — it seems everything has changed. Google announced and immediately shipped “AI Mode,” its answer to OpenAI’s ChatGPT. The company had already implemented “AI Overviews” in 2024, essentially an evolution to its “One Box” feature that provides AI summaries for queries above Google’s familiar list of blue links. According to one study, those summaries are already responsible for a nearly 35 percent decline in outbound clicks — the lifeblood of traditional search engine marketing.

Google knows the question is no longer if AI is replacing search, but rather how quickly. Industry analyst Mary Meeker has an answer: AI adoption, she asserts, represents the fastest change in consumer behavior in the history of digital technology.

With AI Mode, Google has decided to push all in. The company understands that consumers are finding AI chatbots a superior search experience. One recent study found that “more than a quarter of Americans (27%) now use AI chatbots like ChatGPT instead of traditional search engines.” And consulting giant Bain found that “about 80 percent of consumers now rely on “zero-click” results in at least 40 percent of their searches.”

Google knows it’s taking a risk by moving quickly, but no matter what, it must keep its grip on the information-gathering habits of its billions of consumers. That leaves a major question unanswered: If search as we knew it is going away, what will replace search marketing as we knew it? What will supplant the time-honored practices of “SEO” and “SEM”?

If you’ve been exploring this question, you are most likely overwhelmed by a slew of firms, from early stage startups to massive agencies and consulting firms, all claiming an intuitive and familiar answer: “SEO, but for answer engines.”

Some call it “GEO,” short for “generative engine optimization,” and others have dubbed their solution “AEO,” for answer engine optimization. Regardless of nomenclature, the goal is the same: a set of best practices that position a product, service or brand to succeed in the brave new world of AI chatbots.

Chatbots “have been a huge wake up call,” says Pete Blackshaw, CEO of BrandRank.ai, a startup that helps brands adapt to what it calls the “answer economy.” “The trillion-dollar brand marketing industry is being severely disrupted by AI search, and marketers don’t know how to measure it, where to start, or how to win.”

It’s still early, but what’s already clear is that “winning” in AI search means playing a very different game. AI chatbots are the ultimate black boxes – even the companies that run them have no idea how the technology chooses its answers. That means marketers accustomed to precise dashboards of SEM and SEO-driven results must go back to basics. Blackshaw advises his clients to focus on understanding the content ecosystem that informs answer engines’ inputs. The often neglected brand website, for example, “is a fueling station for an AI chatbot,” he says.

Along with similar offerings from companies like Anvil and Profound, BrandRank offers dashboards to help clients understand how their brands are performing inside various answer engines like Perplexity, Claude, ChatGPT, and Google. Once you understand where you stand, you can get to work building out content strategies that might improve your brand’s performance overall.

If that sounds squishy, that’s because in an age of chatbots, it’s harder for brands to hide behind paid marketing. Answer engines scour a massive corpus of data to source their answers, and seem to favor community forums like Reddit or authoritative sites like Wikipedia — neither of which are particularly easy for marketers to influence.

“You can’t control how or when you’re going to be mentioned” in AI answer results, observes Bill Gross, the fabled inventor of paid search, now CEO of ProRata.ai. “There’s no statistics, no reporting.” Gross believes he has a solution: ProRata has developed a platform that builds AI-generated ad units that are contextual to the content of a particular chat. In short, Gross is building AdSense for AI – an approach that allows marketers to buy their way into the AI conversation. But it’s still early days — ProRata has yet to sign a major client like Claude or Perplexity.

In the end, what matters most will be how Google adapts its world-beating advertising services to the emerging experiences of AI chat. For now, Google is treating the various “surfaces” of its AI products as just another channel for its massive AdWords and AdSense businesses. But sometime soon, the company will be compelled to roll out AI-specific advertising units that are purpose built for AI conversations.

What might that look like? On that question, Google has so far remained mute. “Everyone thinks we know the answer to that question,” one source inside Google told me. “We might know more than many,” my source continued, but when it comes to what an AI version of AdWords might look like, an admission: “We don’t have a clue.”

Regardless of whether or not Google has a plan to pivot its $350 billion advertising business toward AI, the fact remains that hundreds of millions of consumers – in particular younger generations – now deploy AI-powered chatbots like OpenAI’s ChatGPT and Anthropic’s Claude to do everything from work-related research to more traditional search behaviors like comparing product reviews or finding new restaurants to visit. Oh, and on the horizon? AI agents that do the searching and possibly even the purchasing on behalf of consumers.

Given the pace and complexity of change, seasoned experts in search advise brand managers to move to a wartime footing. The rise of AI is likely to be “more impactful than the introduction of the Web in the mid 1990s,” said GoFundMe CEO Tim Cadogan, an industry veteran who ran search at Overture and Yahoo, then helmed OpenX during the rise of programmatic advertising. To win the day, marketers must be “extremely observant and agile,” he continued.  “I don’t think any of us know what is going to happen. Whatever happens will happen very quickly. We have to be prepared to throw out the old ways of doing things.”

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

4 Comments on Is AI Killing Search Marketing?

Nearly 30 % of All Bullsh*t Online Is Health Related

 

Every so often I have the chance to catch up with Gordon Crovitz and Steve Brill, the founders of NewsGuard. I was an advisor to their company for several years, and we keep in touch, as we share something of an obsession around the decline of journalism and the related erosion of fact-based information online. Both Crovitz and Brill have long and storied careers in “traditional” media – Brill started American Lawyer and CourtTV and authored countless pieces of long form reporting, and Crovitz was the former publisher of The Wall Street Journal.

NewsGuard launched as something of a bullshit detector during the ascendence of social media eight years ago. The company’s first product created “reliability ratings” of news and information-based websites. Not surprisingly, the company immediately became a target of the far right media ecosystem, and remains one today.

NewsGuard initially covered a few thousand well-trafficked sites, but now covers more than 35,000 sources around the world. Along the way the company has built a tool for advertisers to identify quality inventory for programmatic advertising, several AI-related products, and a database of false claims circulating like microplastics in the information ecosystem’s bloodstream.

As we caught up this week, I told Brill and Crovitz about my work at DOC, which has a mission of finding signal in the increasingly noisy world of longevity science and medicine. “Oh,” said Brill, “did you know that healthcare is the largest category of false claims that we track online?”

I didn’t know, but upon reflection it certainly makes sense. According to Brill, healthcare makes up 931 of the 3,254 provably false claims in NewsGuard’s false claims data base – nearly 30 percent. Why? In one word, Brill told me, money. Hucksters and charlatans have a well-worn playbook leveraging bullshit claims on social media to sell supplements, therapeutics, and services that fail to meet traditional standards of rigorous scientific evidence. The fact that a notorious conspiracy theorist is now in charge of our country’s entire healthcare apparatus certainly isn’t helping.

Brill dedicated a chapter in his recent book, The Death of Truth, to the spread of healthcare-related bullshit. Ten years ago, Brill wrote a deeply reported (and very personal) piece on the mess that is the American healthcare system (it became a best selling book as well). The rise of AI, Brill told me, is only exacerbating the problem.

I’ve been skeptical of the claims made by the tech and AI industry as LLMs and chatbots begin to take over nearly every aspect of the Internet as we’ve known it. My conversation with Brill and Crovitz only deepened my skepticism. When companies like Microsoft claim its AI outperforms doctors, resulting in “a genuine step toward medical super-intelligence,” we tend to scan the headline, shrug, and move on.

But what does “medical super-intelligence” actually mean? Machines are always going to process data faster and more completely than humans. But the interpretation and provision of healthcare is a uniquely human endeavor.  Before we outsource our actions to AI, we best engage in extended and considered debate about the impact of these technologies on our society. If these topics animate your thinking, I hope you’ll consider joining us at DOC, where we’ll be debating exactly these kinds of issues. You can apply for an invitation here.

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

Leave a comment on Nearly 30 % of All Bullsh*t Online Is Health Related

AI Is Breaking The Internet. Can Bill Gross Fix It?

Bill Gross, ocean boiler.

Bill Gross has been here before. 

Back when the Internet was young, when the dot-com wave broke across the monied shoals of Wall Street opportunism, Gross built a world-changing company, took it public, then sold it for billions. 

Google was a fledgling startup with no discernible business model when Gross pioneered a revolutionary approach to Internet advertising now known as “paid search.” It seems obvious today, but in 1998, the notion that advertisers could buy search keywords and only pay if people clicked on their ad – well, that idea was bonkers. 

So bonkers, in fact, that when Gross launched Goto.com at TED in February of 1998, the audience booed and hissed while he was on stage. The gall of this man! This bespeckled, wiry, manic guy from Los Angeles of all places – he wanted to ruin the Internet by putting a price tag on search results!(1) 

Despite the initial response from the TED crowd, Goto flourished, growing through the dotcom crash that devastated most of the early Internet industry. Knowing he could never catch the portals dominating early Internet traffic, Gross pivoted his company to a syndication model, executed a successful IPO, and changed Goto’s name to Overture. By 2002, the company had search monetization partnerships with Microsoft, AOL, and Yahoo – the three largest Internet destinations of the era.

Gross had the right idea at the right time – an idea that Google went on to borrow, improve, and capitalize upon; an idea that drove a profound shift not only in how businesses operate, but also in how our society values, processes, and governs its most fundamental currencies: Information and human attention.(2)

Bill Gross got rich from that idea – he sold his company to Yahoo, his largest customer, in 2003. But if you ask Gross whether things turned out the way he had hoped they would, his answer is a resounding no. Gross surveys the landscape of today’s Internet, and once again, he sees a massive opportunity hiding behind an equally massive problem: Everyone’s using these new AI engines, but so far, no one’s figured out how to make them pay. 

***

Search built the modern Internet. Google organized the world’s information and made it accessible, in the process producing a massive fuel depot that powers yet another revolution: Generative AI. AI’s gluttonous maw – large language models from OpenAI, Google, Meta, Amazon, Claude, and a thousand startups – consume today’s Internet like a pack of ravenous ouroboroi, ripping through the fabric of established business models like “paid search” in the process.  

So of course Gross wants back in, to once again produce an idea so bonkers it could get him booed off the stage at TED(3). But the world has changed dramatically since he last stepped forward as the CEO of an audacious startup. Can Bill Gross do it again? 

ProRata.AI is his answer, the 150th (or so) company Gross has launched through Idealab, a Wonka-esque incubator he founded nearly thirty years ago. Every ten or so years Idealab starts a company so important, so potentially game-changing, that Gross decides to take the role of CEO. ProRata is that kind of company. 

The Internet today is broken, Gross argues. The grand bargain between Internet sites and search engines – we’ll let you crawl our site if you send us traffic in return – that bargain has been subverted by monopolistic Internet giants who refuse to share the wealth, as well as craven startups eager to cash in on the AI boom. And now that billions of consumers have shifted their attention towards generative AI chatbots, the “rest of the web” – the very seed corn that feeds those bots – is withering. 

It’s not that Gross thinks answer engines are bad – quite the opposite. Services like ChatGPT are a “10X improvement” on search, he tells me. “They’re incredible,” he says, “but they need a new monetization model.” 

The model that he built for traditional search won’t work for AI-driven answer engines, Gross explains. And that’s where ProRata comes in. The company’s audacious idea? To build a monetization engine that does for generative AI what Overture did for search some 25 years ago. 

***

Gross needs to solve two problems if ProRata’s vision is to become a reality. First, he has to catalyze a marketplace for publishers’ content – a way for publishers to get paid when AI crawlers ingest and regurgitate content. And secondly, he has to build an advertising solution that works in the context of an answer engine – essentially, he has to figure out what the ads should look like in a world dominated by AI. 

No big deal, right? 

Gross already has help with the first problem. Those ravenous AI bots hoovering up websites at a rate of thousands of crawls a day? They’re shoplifting, Gross says. AI services should pay for the privilege of ransacking the open Internet, he argues. This concept – “pay per crawl” – has already taken root: Internet infrastructure giant CloudFlare has implemented a pay-per-crawl marketplace premised on a similar philosophy. Publishers that aren’t being paid by those data-hungry AI bots can now avail themselves of a free service from CloudFlare that blocks them at the door. 

Now….imagine if all websites blocked AI agents! How might the world change?! Well, those AI services would have no choice but to start paying site owners. But that raises our second problem: AI services need a revenue stream that they could share with publishers – one that scales well beyond the subscription models which currently dominate services like Claude and ChatGPT. “The AI companies are all complaining that they can’t afford to pay,” Gross tells me. “Well, if I can triple their revenue, then they can afford to pay, right?”

Advertising has always been the lifeblood of the Internet. With ProRata, Gross has built an advertising solution that generates ads on the fly based on the context of each users’ queries. To catalyze his marketplace, ProRata operates an AI answer engine called Gist.AI that features content from licensed partners who are paid each time the Gist service uses the partner’s content to generate an answer. Gist also serves as a proof of concept for ProRata’s advertising solution, which splits revenues with its publishing partners. Here’s what it looks like:

That ad for the EV in the middle of the page? That’s ProRata’s revenue model in action.

Gross knows that Gist.ai will never get to the scale necessary to compete with the ChatGPTs of the world. So just as he did with Overture, Gross is leaning into a syndication model – distributing Gist “answer bots” across publisher sites, signing up advertisers, and working to sign monetization deals with the AI answer engines of the world. 

If Gross has his way, a significant number of both publishers and AI chatbots will avail themselves of ProRata’s services, creating a marketplace similar in nature to the one Goto and Overture created 25 years ago.(4) 

I was fortunate enough to pay attention to Gross and when he launched Goto back in 1998 – I later devoted a chapter in The Search to his story. Gross doesn’t have to sign Google Gemini or OpenAI to succeed – if he garners even one-tenth of the current AI answer engine market, he’ll have built a company far bigger than Overture, and potentially just as impactful. 

Will ProRata prove to be Gross’s second act? I’d not bet against it – and if ProRata’s model takes root, it will mark a significant departure from how the Internet has worked to date.  More than two decades after sparking a revolution in search, Bill Gross is once again driving a controversial and counter-intuitive startup, one designed to equitably monetize a massive shift in technology, capital, and consumer behavior. Keep an eye on ProRata. I smell a good story brewing. 

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

  1. Back in the day, TED was a very different kind of conference. Its audience had a strong anti-commercial vibe – pitching from the stage was seen as declassé, and every so often a speaker would endure boos or hisses if their talk offended the community’s sensibilities. I happened to attend those early TEDs, and I launched The Industry Standard at the same 1998 gathering where Gross was booed. In the room were not only Gross and I, but Jeff Bezos, who has a cameo in Goto’s launch narrative, as well as Sergey Brin and Larry Page of Google. 
  2. Google launched the same year as Goto – but with a decidedly anti-advertising philosophy. “We expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers,” wrote Google founders Sergey Brin and Larry Page in a research paper that launched Google. “the better the search engine is, the fewer advertisements will be needed for the consumer to find what they want.” As Google grew and became a consumer destination in its own right, Gross held talks with Brin and Page, pitching either a merger or a commercial partnership. Instead, Google’s founders decided to emulate and improve on Gross’s model.
  3. Alas, TED only lets people give standing ovations these days….
  4. Of course, there’s one big difference: The Internet market today is orders of magnitude larger than it was in 2003. 
5 Comments on AI Is Breaking The Internet. Can Bill Gross Fix It?

How Google Can Win the Future

[Second in a series, first post here]

This past week, Wall Street caught up with the rest of us and realized that Google has lost its monopoly grip on search. The trigger wasn’t Google losing an anti-trust case – that happened last summer. Nor was it the first ten days of Google’s ongoing search remedies trial. Instead, it was a statement just two days ago by an Apple executive, Eddie Cue, which led to an almost instantaneous panic amongst investors.

Cue told the court that consumers’ preference for using AI agents had led to a decline in search traffic inside Apple’s Safari browser (Google pays Apple more than $20 billion a year to secure that traffic – a major focal point of the government’s case).

Cue then twisted the knife: A decline in Safari’s Google traffic, he said, “has never happened in 22 years.”

A near perfect storm of “never happened” has settled over Google this past year: Twin rulings labeling the company a monopolist, a 10 percent annual decline in its stock compared to a 10 percent increase in the NASDAQ index, and the rise of generative AI as a replacement for search, the first significant competitive threat to the company’s 25-year run as the dominant force in how humanity asked questions of the world.

Poor Google. The company finds itself in the unenviable position of IBM and AT&T in the ’70s and ’80s, or Microsoft in the early 2000s. It’s become an industry titan beset by government antitrust actions that hobble its ability to respond to fundamental and permanent market changes.

What on earth can Google do? Well, I’ve got good news for the company: There’s a way out, but it’s going to hurt a little.

***

To understand the actions Google might take to ensure its continued membership in the Magnificent Seven, it’s worth taking a step back and studying Microsoft. Throughout the ’90s, Microsoft led the “Four Horsemen” – blue-chip tech stocks which drove unprecedented market returns during the first Internet boom (the four were Microsoft, Intel, Cisco, and Dell, if you can believe it). But by 1999, Microsoft was ruled a monopoly just as the center of the technology industry shifted from desktop PCs to a mobile-centric world built on top of the Internet.

For nearly two decades, Microsoft had dominated tech. Its Windows operating system captured north of 90 percent market share. Windows was the world’s #1 consumer-facing technology brand – the face of an industry. But once Microsoft settled with the US government and ended its five year anti-trust fight, the company’s brand, stock, and fortunes began a decade of decline. Founding CEO Bill Gates stepped down, and in a disastrous strategic misstep, newly installed CEO Steve Ballmer doubled down on the company’s past by putting Windows at the center of everything the company did. Here’s the Wall Street Journal’s summary of  the Ballmer era, a piece written as incoming CEO Satya Nadella was preparing to take the reins:

“In many cases, Microsoft latched onto technologies like smartphones, touchscreens, ‘smart’ cars and wristwatches … long before Apple or Google did. But it repeatedly killed promising projects if they threatened its cash cows [Windows and Office].”

Let’s update that paragraph, written in 2013, for present-day Google:

“In many cases, Google latched onto technologies like artificial intelligence long before Apple or OpenAI did. But it repeatedly killed promising projects if they threatened its cash cows [Search and Advertising].”

The lessons of Microsoft don’t end in 2013 – in fact, the company subsequently managed one of the most impressive turnarounds in modern business history. Acknowledging Ballmer’s mistakes, Nadella  deprioritized Windows and focused Microsoft on a decidedly less sexy but far more rewarding market: Enterprise infrastructure. By 2020, Microsoft was once again one of the most valuable tech companies in the world. Windows still existed, but as tech analyst Ben Thompson declared, it was no longer a driver of Microsoft’s business. Microsoft is now a leader in cloud-based infrastructure – an arms merchant selling highly profitable services to any and all comers.

***

Might Google learn from Microsoft’s stumbles and subsequent pivot? Indeed. As I hinted in my last post, it’s time for Google to release its death grip on consumer-facing search, and embrace a future where its cash cow becomes a commodity driving a new ecosystem of innovation. In that post, I wrote:

Google should make the database of intentions freely available to anyone who wishes to license it. 

In the remedies phase of Google’s search trial, which wraps today, the government has already demanded that Google share its core search data with any and all comers. Google has responded by taking to the fainting couch, declaring that such a remedy would “allow anyone to completely reverse engineer, end to end, every aspect of our technology stack.”

But would that really be so bad? At the end of my last post, I promised to lay out why Google sharing its search data could be a net win for not only Google, but the entire tech industry, and by extension, the world. Here’s my step by step guide:

Step 1. Negotiate a settlement with the government that embraces Google’s sharing of search data. If it acts quickly and keeps a long view, Google could negotiate the right to both charge market rates for its “search as a service” – which should bring in handsome profits – while continuing to offer its current search services (including Gemini AI). This would allow Google to continue to milk its consumer facing search application while also profiting from new applications built on top of its core search index and data.

Step 2. Design and launch a technology platform that delivers search as a service to qualified developers around the world. This platform would give partners access to Google’s search index in real time, and let them build whatever they chose to build on top of it. Like Microsoft, this would position Google as an arms merchant, selling high quality commodity search to all comers. Platform policy should include bi-directional data sharing, such that new services that leverage Google’s index would share engagement data back to the Google service, federating incremental improvements across all partners in the ecosystem.

Step 3. Boldly, loudly, and confidently begin to tell a new narrative about Google’s future.  Just as Microsoft was once a “Windows” company but moved past its history to become an enterprise winner, Google is evolving beyond search to become a neutral platform company that is accessible to all. It already has a robust asset to drive that journey in Google Cloud Platform, and this move will supercharge that business. Now the company will be single handedly responsible for unleashing a data asset – literally the database of intentions – that will create thousands of new companies and services, and with that tens of thousands of new jobs across the technology ecosystem.

Step 4. Continue to run and improve Google’s current search and AI services on top of this new platform. Wall Street hates abrupt change, and it’ll take years for AI and chatbots to eclipse traditional search in terms of revenue and profit. Don’t worry, Wall Street! Just as Microsoft kept selling Windows and Office, Google will keep selling search too.

Step 4. Imagine and build new services to compete on top of its own services. With consumer search – owning the customer at the interface layer – no longer the future of Google, Inc., the company will be free to compete with OpenAI, Perplexity, Kagi, and the thousands of new startups that will undoubtedly spring up once Google’s core search service becomes available as a platform. Instead of exhausting itself attempting to build a walled garden around search (and awkwardly shoehorning Gemini into its increasingly unwieldy core service), Google will be once again be a place where big ideas and crazy moonshots can be taken at scale.

Will this work? Of course there are obstacles, but I think it could. And it certainly sounds like a more appealing future than one of constantly defending yesterday’s business model of walled gardens and fealty to advertising economics. In one bold move, Google could unshackle itself from government oversight as well as a mirthless, spiraling death loop of enshittification.

In the end, Google turning search into a service would benefit the world in a way that would once again position the company as an innovator driving value across all of society – kind of like it was when I wrote The Search 20 years ago. Let’s face it: Google search has gotten pretty shitty over the past 20 years – and everyone’s noticed. If we’re headed into a world where AI chatbots become our interface to knowledge, the next generation of Claudes, Bings, and ChatGPTs will desperately require a high-quality search index that they, and we, can actually trust. That index could and should be Google’s. But only if the company has the courage to let go of its past.

—

Special thanks to the folks at Kagi, and in particular this blog post, which lays out the case for Google to share its database of intentions. 

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

2 Comments on How Google Can Win the Future

Free the Database of Intentions: Could Google Thrive If It Gives Away Its Data?

Over the past 25 or so years, I’ve argued that Google has built a massive database of intentions – the aggregate result of every search ever entered, every page of results ever tendered, and every path taken (there’s a lot more to it, but that’s the key stuff). I’ve tracked this extraordinary artifact since 2003, and have come to believe that Google’s control over it has become a inhibitor to innovation and flourishing in our society.

The US government – yes, even this one – agrees with me. In the nearly three decades since Google first launched, the company has gone from champion of the open Internet to established monopolist whose principle business is protecting its profits. With the advent of consumer AI, that principle business is imperiled. Google is protecting a revenue stream that it must understand is no longer defensible, either by law or by practice.

Instead of innovating in its approach and business model, Google is digging in its heels and hoping it can exhaust the government through years of appeals, half measures, and delay. It’s a well worn playbook, and it’s bad for everyone, including Google.

But there’s another way, one that would re-establish Google as the innovator it once was. It’s bold and crazy by most traditional business standards, but it’s most likely the only way the company avoids a fate of gradual decline and irrelevance. And it has the added benefit of getting the government off the company’s back.

Google should make the database of intentions freely available to anyone who wishes to license it. 

The tech industry is in the midst of yet another great turn of the wheel. The old interface to knowledge and compute that has served Google so well – command-line queries and a list of blue links – is in the process of being ceded to natural language, multi-modal conversations driven by generative AI. Google knows this, which is why the company has cautiously integrated its Gemini technology into its core search results. Sooner than it would like, search as we know it will become a commodity, a critical but largely forgotten layer of complexity that will be buried – like DOS with Windows or Unix with iOS – under layers of new services in the process of being invented.

These services could be wondrous, they could be precise and accurate, they could be dizzyingly profitable, and they could change how we understand the world. But without high-quality search to enable them, they will never exist.

Google could not only insure these new services thrive, it could take credit for establishing a platform that once again changes the world. And along the way, it could loose the chains of government oversight, be paid handsomely for its service, and even build its own applications on top – but only if the company has the courage to change.

***

Two separate courts have found Google to be a monopolist in both its search and its advertising businesses. In search, the company is now in what is called the “remedies” portion of the trial. Remedies are a strange beast – now that the court has labeled the company a monopolist, the two sides spend the better part of a year arguing what to do about it.

If Google were a recalcitrant and unruly child and the government an exasperated parent, we’re in the awkward part of the family debate where appropriate punishment is being discussed. Google wants to be sent to its room without dinner. The government wants to send Google to a military reform school for the rest of its natural life.

You’ve probably seen the headlines: The government is asking Google to:

    • Divest its Chrome browser and possibly its Android business
    • End its practice of inking exclusive distribution search agreements with partners like Apple and Mozilla
    • License its core search data (what I call the database of intentions) to competitors

Like a kid hoping for a slap on the wrist, Google rejects each of the governments proposals, suggesting instead that it simply modify its current practices to allow for more competition in what previously were exclusive deals for search distribution. Divestiture is a non-starter, Google argues. And licensing its search data to others would be a mortal blow to the company, making further investment in the business “inviable,” in CEO Sundar Pichai’s words.

The Times further  reports that Pichai called “forced data sharing a ‘de facto divestiture’ of the company’s intellectual property that would ‘allow anyone to completely reverse engineer, end to end, every aspect of our technology stack.'”

Pichai is almost certainly right. Search is a notoriously difficult and expensive enterprise to establish and maintain – Microsoft has spent hundreds of billions attempting to compete with Google, and it’s largely failed to take share. That’s because Google’s control over a real-time database of intentions allows it to keep its advertisers, consumers, and partners locked into a closed-loop business ecosystem where it can reap the lion’s share of profits.

Barring settlement, which seems unlikely, the smart money is betting that the judge in the case will not force Google to divest either Chrome or Android, but will force the company to both cease its exclusive distribution deals, as well as license its core search data to competitors.

Given Google has already signaled a willingness to modify its distribution deals, the most interesting aspect of this ongoing drama now centers on the database of intentions.

The government wants to break Google’s grip over this critical asset.  Google is signaling it will fight to the death to avoid sharing it. But what might happen if Google stops fighting the future, and instead embraces it?

That’s a question worth at least one more post. I’ll link to it here once it’s finished. UPDATE: Here’s that second post.

—

You can follow whatever I’m doing next by signing up for my site newsletter here. Thanks for reading.

 

 

1 Comment on Free the Database of Intentions: Could Google Thrive If It Gives Away Its Data?