What share of sites serve an llms.txt?
6 published answers, 5.86% to 47.49%. Issued 2026-11-02.
The answers run from 5.86% to 47.49%, a spread of 8.1x, and none of it is error. Three parameters decide the number: which population was sampled, whether an HTTP 200 was validated as a real file, and how unreachable hosts are treated in the denominator. Every measurer below states its own parameters; the fault is in quoting a single figure as the rate, not in the measuring.
This page does not rank these measurements by quality and does not claim any of them is wrong. Each is correct for the population and rule it declares.
8.1×between the lowest and highest
6published answers, same question
3parameters that decide the number
The answers, side by side
| measured by | population | n | rate |
|---|
| Thunderbit | Tranco top 10,000, list downloaded 2026-05-06 | 10,000 | 5.86% |
| Seomator | Cloudflare Radar top 100,000, requested 2026-09-13 | 100,000 | 8.60% (11.4% of responders) |
| Rankability | Tranco list V3YPN, checked 2026-09-18 | 10,000 | 8.30% |
| Ahrefs | all 137,210 domains in Ahrefs Web Analytics that received traffic in May 2026 | 137,210 | 28.00% |
| Atlas (this index), brochure corpus | the brochure corpus — enterprise and mid-market marketing sites in our live rotation | 197 | 42.13% (95% CI 35.45–49.11) |
| Atlas (this index), agent-native corpus | the agent-native corpus — hosts that publish an agent-facing interface | 1,754 | 47.49% (95% CI 45.16–49.83) |
Every row was read at its own source on the date given in the record below, not
quoted from a summary of it — and each name links to that source. The machine-readable version of this page, with each URL, access
date, validation rule and denominator, is
llms-txt-reconciliation.json.
Three parameters decide the number
1. Which population. A ranked list of the largest sites, a set of domains
that pay for analytics, and a set of hosts that already publish an agent-facing interface are
three different webs. They are not samples of one population with different noise.
2. Whether a 200 was validated. A request to /llms.txt that
returns HTTP 200 has not found a file. Two of the measurements below quantify the gap:
- Thunderbit — 1,606 responses returned HTTP 200 at /llms.txt; 586 passed validation. 618 were off-target redirects and 367 generic HTML. Counting every 200 would overstate adoption by 2.74x.
- Seomator — 8,598 domains publish a file; 20,171 returned a 200 at that path, of which 10,775 served ordinary HTML. Counting every 200 would overstate adoption by 2.35x.
3. How unreachable hosts are counted. Seomator publishes both: 8.6% over the whole list and 11.4% over the 75,353 that responded — a factor of 1.33 from the denominator alone, same team, same crawl. Rankability keeps every sampled domain in and reports the unknowns beside the rate. This index drops a host that returned nothing classifiable, counting it neither as present nor as absent.
We can produce the spread ourselves
One instrument, one day, two declared populations: 42.13% of 197 hosts (brochure corpus) and 47.49% of 1,754 hosts (agent-native corpus). The difference is the population,
not the measurement — which is the whole argument of this page, made where we cannot blame
anyone else for it.
⚠️ These two figures are not published anywhere else, so unlike every other row they cite no
prior article. Their citation is the method and the chain instead, and the record gives the
exact estimator call that returns each one.
What each measurer says about its own figure
- Thunderbit
- “status codes are a poor proxy for adoption”
- Seomator
- “a checker looking only for 200 OK overcounts adoption by more than double”
- Rankability
- “Read the denominator. These percentages include every sampled domain, even when a site blocked the scan or could not be reached.”
- Ahrefs
- “Ahrefs Web Analytics customers skew more technical and SEO-aware than the web at large, so treat the 28% adoption figure as an upper bound.”
- Atlas (this index), brochure corpus
- “A rate is a rate for the declared target set. It is a sample, not a census.”
- Atlas (this index), agent-native corpus
- “Measurement is from a single vantage. A series that mixed vantages would mix WHERE with WHEN.”
These are quoted because they are the point. Every party here publishes the limits
of its own number; a reader who takes one figure out of that context is not being misled by the
measurer.
What is not in the table, and why
Three figures were excluded while this page was being assembled. Two of the exclusions are
our own errors, and they are listed here because a page about reading sources carefully is worth
nothing if it hides the times we did not.
- ~2% (2025), attributed to the HTTP Archive Web Almanac — seen only as a citation inside a Common Crawl post, not verified at the Almanac itself. A page about reading sources does not get to quote one at second hand.
- 5.61% for the top 10,000, which an earlier internal note attributed to the HTTP Archive — the figure appears in a Topify post with no named source, and no HTTP Archive publication for it could be found. The attribution was ours and it was wrong.
- 15.8% over 549 reachable domains, which an earlier internal note attributed to Rankability — no Rankability surface publishes it, live or in the 2026-10-18 archive snapshot. Their published rule is the opposite: all domains stay in the denominator. The arithmetic was most likely ours, recorded as if it were theirs.
Method, and the right of reply
Nothing here re-measures anyone. Each external figure is the author's own published number,
read at the author's own URL on the access date recorded. The only new measurements are the two
marked Atlas, computed from our signed append-only chain on the issue date; they move as the
corpus is re-observed, and the date is part of the claim.
Every named party received this page in draft on 2026-11-02, with its figure and our characterisation of its method, before it was published. Thunderbit replied and confirmed every figure attributed to it, and asked for one change, which is in. Seomator, Rankability and Ahrefs had until 2026-11-06 and did not reply, nor in the days since. Their rows stand as read at their own sources on the access dates recorded. A correction from any of them will be made and shown, as will any correction from a reader.
If something here is wrong about your work, write to
info@geosensor.tech and it will be corrected with the
correction shown.