Earlier this week I posted something of a coda for the open web, but today news broke of a small victory for “how things used to be.”
The backstory: For decades, Google has been the beating heart of the Internet’s ecosystem, pumping the oxygenated lifeblood of traffic across tens of millions of revenue-starved websites. That’s all changed over the past few years as Google has erected a massive set of AI-driven walls around its core search service. As I said in “Google Encloses the Web,” Google no longer views the web as a destination, but rather as “raw material” for the company’s core product, which, like so many of its peers, is attracting and retaining human attention it can sell to advertisers and, via Gemini subscriptions, to businesses and consumers.
Regardless of Google’s recent moves, there remains a cottage industry of small firms that earn their keep by helping businesses make sense of the search ecosystem. These firms build databases of Google results, then sell that data to a large swath of the Internet economy. The practice has always lived somewhat in the gray – Google’s internal policies forbid the bulk gathering of Google’s own search results. Like Google its peers Amazon, Meta, Netflix, and many others, Google expends herculean resources to defend those policies. And until recently, the question of whether Google or its data-scraping adversaries have legal standing has not been put to the test.
SerpApi is one such adversary. It has built a business by scraping Google’s results and packaging them for clients interested in understanding the trillion-dollar ecosystem that Google commands. On its site, SerpApi claims Uber, KPMG, Shopify, and NVDIA as customers, among many others. Late last year, Google decided it had enough, and sued SerpApi for “running an “unlawful” operation that bypasses Google’s security measures,” essentially selling “a back door to Google’s proprietary search engine.”
At issue is whether or not search results – and the queries that drive them – constitute public data, or if Google can claim proprietary ownership. SerpApi claims it is “backed by the First Amendment” and the Fair Use Doctrine, but until now, its arguments have been mostly academic.
Yesterday, a U.S. District Judge handed SerpApi a clear win, dismissing Google’s suit and claiming the search giant did not have standing. The details of the ruling are worth considering: Google claimed SerpApi was bypassing Google’s anti-bot security protocols, in violation of the DMCA, a 1996 law that, among other things, makes it illegal to circumvent technologies that protect copyrighted material.
The judge determined that while SerpApi most likely did use circumvention to scrape copyrighted information – it’d be impossible not to, given the content of Google search results – Google could not claim it was acting on the actual copyright holders’ behalf by enforcing its anti-scraping policies. Put another way, Google has no standing to claim the anti-circumvention protections of the DMCA, because Google is, in essence, the biggest scraper of them all.
Why does this matter? It may not, given the size and scope of Google’s legal team and its lobbying muscle in DC. But it is a fascinating precedent given the deteriorating relationship between Google and actual copyright holders like Dow Jones, Axel Springer, and CNN. In a recent Wall Street Journal article, all three (plus many other publishers) were reported to be considering abandoning Google’s index altogether, in large part because of Google’s decision to enclose the web. “Turning them off and blocking them entirely is 100% on the table,” People Inc. Chief Executive Neil Vogel told the Journal.
As the web shifts from open to closed, from intelligence at the edges (us), to intelligence as a service (the AI providers), it’s worth paying attention to the little stories that might just point to a new architecture of control on the web. The SerpApi decision is just such a story.
—