The relationship between publishers and the large technology companies that redistribute their work has always had the quality of a difficult marriage — periods of cooperation interrupted by arguments about money, credit, and who exactly is benefiting from whom.
Now Cloudflare, the site security company whose software protects a considerable swath of the publishing industry, is handing one side a new tool in those ongoing negotiations. Beginning September 15, the company's software will automatically block the crawlers that scrape websites for both search indexing and artificial intelligence training. The change applies to new publishers joining the service and those using its free tier.
The companies affected by this default setting include, notably, Google and Microsoft's Bing — which is to say, the two search engines that most humans actually use.
For publishers including The Financial Times, Condé Nast, and The Atlantic, all of which use Cloudflare's services, the change represents something between a bargaining chip and a castle wall. The question of whether their journalism should be used to train AI systems without compensation has moved from theoretical grievance to operational policy.
(Whether individual publishers will flip the setting back on, having calculated that the traffic from Google outweighs the principle of the thing, remains to be seen.)
The move arrives during a period of considerable legal activity around these questions. Cloudflare is now the fifth major supply-side platform in the last year to sue Google over what it describes as anti-competitive practices in digital advertising — a roster that suggests the industry's patience with certain arrangements may be wearing thin.
Meanwhile, the advertising industry's nonprofit coalition has found itself reaffirming a commitment to free speech, following allegations that it coordinated boycott efforts against X, the platform formerly known as Twitter.
It is, in other words, an ordinary week in the content business, where the fences keep moving and everyone is still trying to figure out who owns what.
Original story published in adweek.com: "ADWEEK Tech Advantage: Publishers Pull Back from Google AI Search, Creators Push On"