Pudels Kern · Part 4 — published on 10 August 2026
Bots are the majority of the web. On 15 September that becomes a business model.
On 3 June 2026, Matthew Prince, co-founder and chief executive of Cloudflare, put a number into the world: 57.4 per cent of requests for web content across his network come from automated systems, 42.6 per cent from people. For the first time in the history of the internet, machines hold the majority. Prince had expected that point at the end of 2027, and his reaction has been quoted ever since: it happened faster than he predicted.
I read the news like everyone else. Then I looked at what else had been measured on the way there — and at what has happened since. What struck me is that the number is the least interesting quantity in this story. The interesting one is in the calendar.
What was measured
For a subject this new, the evidence is surprisingly thick.
HUMAN Security analysed more than one quadrillion interactions from 2025 for its 2026 State of AI Traffic & Cyberthreat Benchmark Report (26 March 2026). Automated traffic grew by 23.51 per cent, human traffic by 3.10 — a factor of eight. Traffic from AI agents and agentic browsers grew by 7,851 per cent. More than 95 per cent of it sits in three sectors: retail and e-commerce, streaming and media, travel and hospitality.
Cisco measured the shape of the traffic rather than its volume in AI Impact on Wide Area Networks (May 2026). An agent generates up to 450 per cent more traffic than a human for the same task. Inference connections run about twice as long as ordinary web transactions — a median of 1,292 against 643 milliseconds. By 2035, Cisco expects AI inference to account for a quarter of all network traffic.
Adobe Analytics supplied the commercial side in its Q2 report of 16 April 2026. In March 2026, visitors arriving at US retail sites from AI assistants converted 42 per cent better than visitors from other channels. Twelve months earlier the same channel ran 38 per cent worse. The sign flipped in a single year.
These numbers hold. I checked each one at its source before writing it down. Which is precisely why I can also say what they are not.
The first second look: the number is a definition
Perplexity publicly disputed Cloudflare's analysis, arguing that it lumps user-initiated fetches together with automated scraping. Prince himself called the underlying classification messy. NBC counted 57.4 per cent; Cloudflare elsewhere said 57.5. Imperva's Bad Bot Report put 2025 at 53 per cent — a different basket, because it counts API and app calls too.
Behind all of this sits a question with no technical answer. When an agent fetches a page on my behalf, because I asked it to — is that me, or is that a machine?
That is a commercial decision dressed as a metric. Whoever gets to make it also helps determine who may claim reach, who pays for access, and whose ad impression counts.
The second second look: everyone measuring has something to sell
Cloudflare sells bot management and billing models for crawlers. HUMAN Security sells defence against synthetic traffic. Cisco sells network capacity. Adobe published the 42 per cent around the market introduction of its LLM Optimizer, and the data comes from its own platform, audited by nobody.
None of that devalues the measurements. They are large, their methods are described, and they are the best evidence available. It does explain why all four point the same way and none the other. Four independent studies with a shared commercial interest are not a chorus — they are four soloists reading from the same score.
The sober reading: the direction holds. The magnitude is provisional.
What has already happened
The argument over the decimal point would be harmless if the effect were still ahead of us. It no longer is.
Traffic to content businesses is falling away. Google search referrals to publishers worldwide fell by around 33 per cent in the year to November 2025, and by 38 per cent in the United States. Ahrefs measured across 300,000 search terms in February 2026: where an AI summary appears above the first result, that result loses roughly 58 per cent of its clicks. Pew Research counted an 8 per cent click rate with an AI summary against 15 per cent without.
The clearest single case is Wikipedia. The most-cited domain in Google's AI answers lost around 8 per cent of its human page views in 2025. Being cited and being visited have become two different things.
It is already in an income statement. In Alphabet's quarterly figures of 29 April 2026, Google Network advertising revenue — the part that runs across third-party sites — fell 4 per cent to 6.97 billion dollars.
And the other side is live too. Visa, Mastercard and Stripe brought payment rails for agents into service during 2026; in January 2026 Google presented a commerce protocol at the NRF, co-developed with Shopify, Etsy, Wayfair and Target. Mastercard and Santander completed the first end-to-end agent payment inside a regulated European banking framework. The agent no longer only reads. It pays.
Between those two paragraphs sits the whole movement: on one side the visitor disappears, on the other the buyer appears, and they are not the same event.
The bargain that is breaking
Read 7,851 per cent and the instinct is to extrapolate. That instinct is wrong, because the share is the uninteresting quantity. What matters is the bargain coming apart behind it.
For thirty years the deal was: you take my content, you send me a visitor. Cloudflare has now put a figure on that exchange. Anthropic's crawler fetched roughly 38,000 pages for every single visitor it sent back; OpenAI's ratio stood at about 1,091 to one. By June 2026, training crawlers accounted for 50.6 per cent of AI bot traffic on Cloudflare's network, and search bots for just 10.7 — the one variety that historically paid its way in clicks.
Joint work by Cloudflare and ETH Zurich, published in April 2026, shows how deep this runs. More than 90 per cent of the pages large-scale crawlers process are unique in content. The caching layer of the web is built on the assumption that popular pages get fetched repeatedly. Machines do not read what is popular. They read everything, once.
That is why Cloudflare is acting. And because the result takes effect in six weeks, it is worth looking closely — what happens there is narrower, and more consequential, than the headlines suggest.
What actually happens on 15 September
Cloudflare sits as a doorman in front of a large share of the world's websites. Every bot that knocks announces who it is. Today the door stands open to almost anything identifying itself as a search engine.
The difficulty is mixed-use crawlers. The same bot fetches a page for search results, for model training and for an agent — in one indistinguishable operation. In its announcement of 1 July 2026, Cloudflare named the world's largest search engine explicitly: it obtains roughly twice as much information as other AI companies, because a site can hardly stay discoverable in search without also being used for the AI product.
From 15 September, Cloudflare changes the doorman's default answer for exactly these mixed-use crawlers. The purpose is a demand addressed to the AI companies: separate your bots. Run a clean search crawler and a clean AI crawler, and the search crawler still gets through.
The corridor is narrow. The new default applies only where four conditions meet: the site sits behind Cloudflare; the page in question carries advertising; the account is a free-tier account, a new customer or a newly created site; and the operator has never touched the setting.
In parallel, Cloudflare is replacing billing per fetch with billing per use. Payment follows content being used in an answer, rather than a page being retrieved. The first partners are Ceramic.ai and You.com.
Who this actually hits — three cases
The news site: fully affected, and deliberately so. This is the intended party. Ad-funded pages are precisely the case the rule was built for. Two things change. Load falls away — by Cloudflare's own data, more than 50 per cent of AI crawl traffic goes on re-fetching unchanged pages. And a revenue path opens through billing per use. The price is a risk: if the AI companies decline to separate their crawlers, the site drops out of their answers. That is the wager.
The open online retailer: the automatic change mostly fails to fire. A shop rarely carries third-party advertising on its own product pages. The second condition drops out, and on 15 September nothing happens. The retailer's issue arrives earlier and runs larger: retail is one of the three sectors holding more than 95 per cent of AI traffic. The machine share is already in the house — usually without appearing in the figures.
The closed B2B shop: least affected, most exposed. Behind the login no bot sees anything, so nothing changes there. What decides the outcome is the public frontage: product overviews, data sheets, technical descriptions, references. That is exactly what an agent reads when a buyer asks who supplies part X to specification Y. Those pages carry no advertising, so the default never applies. The exposure runs the other way: such frontages are frequently unreadable — data sheets as PDF only, figures behind a login, pages that stay blank without JavaScript. They are not blocked. They are simply not found. For the outcome, that is the same thing.
What this means for the customer
A buyer asks an assistant instead of calling three suppliers. The assistant answers from what it can read. As a growing share of the web becomes blocked, priced or plainly unreadable, the answer narrows — and it favours whoever stayed readable, or paid to be.
For the customer that means less choice, experienced as no choice at all. He sees what the agent names. Which is why going unnamed does not feel like exclusion to him. It feels like not existing.
The heart of the matter: the denominator tells you first
Where does a business notice this internally? At a place nobody treats as strategic.
Conversion rate is a fraction. Orders on top, visits underneath. Once machines are counted underneath, the figure stops being a statement about customers and becomes a statement about customers plus compute — in a ratio nobody knows.
Everything hanging off it inherits the problem. Bounce rate: an agent that lifts the opening hours in two seconds and leaves has completed its task; booked as an abandonment, it looks like a fault on the site. Time on page: a blend of human reading and machine collection. A/B tests: one variant can win because it is more machine-readable, and the result gets read as a statement about taste. Cost per visit: calculated on a denominator nobody has examined.
That is the heart of the matter. A thirty-year bargain is being cancelled and repriced — and the instruments most businesses steer their digital operation with report none of it, because they write both kinds of traffic into the same number.
Three steps, in this order
Separate first, then decide. Before any strategy gets adjusted, the traffic needs separating: server logs, user-agent analysis, the data from your own provider. While the denominator is unexamined, everything derived from it is a guess with a decimal point.
Readability is infrastructure. Structured data, clean server responses, short load times and clear page structure decide whether a machine can use your content at all. Adobe measured around a third of home-page content in US retail as unreadable to AI models. That is IT work rather than a campaign, and it belongs in the plan and the budget accordingly.
A default is a decision — including the one you never made. Before 15 September, it belongs on the table which crawlers the business admits, which it prices and which it refuses. All of it is legitimate; all of it has a price. And because a mid-market business now stands on both sides — it publishes content and it sends out agents of its own — that judgement belongs at the same table as the decision about sales channels, rather than in a configuration file.
To close
I have worked on digital business models for over 25 years, and I have watched more than once as a new technology was first debated as a question of strategy and turned out, months later, to be a question of measurement. The order is rarely the other way round.
Before adjusting the business to a new reality, examine the instruments you see it with. Then check the calendar.
Sources
- Cloudflare Radar; figures from Matthew Prince, 3 June 2026; Perplexity's objection, reporting from June 2026
- Cloudflare announcement of 1 July 2026 (TechCrunch reporting, 1 July 2026): default blocking of mixed-use crawlers on ad-carrying pages from 15 September 2026, applying to new customers, new sites and free-tier accounts; move from Pay Per Crawl to Pay Per Use with Ceramic.ai and You.com as first partners
- Cloudflare publications on crawl-to-refer ratios since August 2025
- Cloudflare / ETH Zurich, study on caching behaviour, April 2026
- HUMAN Security, 2026 State of AI Traffic & Cyberthreat Benchmark Report, 26 March 2026
- Imperva, Bad Bot Report 2026, 29 April 2026
- Cisco, AI Impact on Wide Area Networks: Cisco Report 2026, May 2026
- Adobe Analytics / Adobe Digital Insights, Q2 2026 AI Traffic Report, 16 April 2026 (calendar Q1 2026, US retail)
- Press Gazette (decline in search referrals), Ahrefs (February 2026, 300,000 search terms), Pew Research (click rates)
- Alphabet, quarterly figures of 29 April 2026
- Reporting on Visa Intelligent Commerce, Mastercard Agent Pay, Stripe Agentic Commerce Suite and the commerce protocol presented at NRF in January 2026
Michael Kraewing · Interim Manager