Specimen assay AS-2026-11-04

Issued 2026-11-04T09:00:00Z · this is the whole deliverable, not a summary of one

What share of this population took a stance on AI agents in its robots.txt, as at the declared date?

Of 639 hosts in the declared population, 152 took one: 23.79%, 95% confidence interval 20.65%–27.24%, as at 2026-11-04T09:00:00Z.

Everything needed to disagree with that number is on this page: the population and its denominator rule, the window, the evidence count, what was excluded and why, and the one command that recomputes it from the signed chain.

Request a scope What an engagement costs

23.79%took a stance
20.65–27.24%95% interval (Wilson)
18,932signed records behind it

What was measured

Capability class PH-C005 / PH-C006 — whether a site's robots.txt takes any stance on AI agents — naming one, or blocking one. PH-C005 is silence on AI agents; PH-C006 is blocking an AI-era agent.

The instrument is the robots_stance probe. The question asked of it: What share of this population took a stance on AI agents in its robots.txt, as at the declared date?

The population, and why it is not 678

Declared as: hosts in this index's live rotation — agent-native hosts and the brochure corpus — that were observed inside the freshness window.

678 hosts produced an observation inside the window. 39 of them returned nothing we could classify — a transport failure, not an answer — so they are recorded unknown and counted neither as present nor as absent. That leaves 639 hosts in the cross-section.

Absence of a machine-readable signal is an upper bound on absence of the capability, never proof of it. Where the instrument cannot tell the two apart, the record says unknown and the host leaves the denominator.

The window

A host is in the cross-section only if it was observed within 7 days of 2026-11-04T09:00:00Z

oldest observation counted2026-11-01T21:35:06Z
newest observation counted2026-11-04T07:51:25Z
median observation2026-11-04T03:43:17Z

The result

took a stance152
of hosts counted639
rate23.79%
95% interval20.65% – 27.24%
interval methodWilson score interval, z=1.96
of which, naming an AI agent96
of which, blocking one56
minimum n before a figure is withheld30

The evidence

18,932 signed observation records lie inside the window, chain sequence 882,249 to 1,130,950: 15,072 absent, 2,687 present, 1,173 unknown.

More records than hosts, because a host in rotation is observed repeatedly; the estimator takes one cell per host — the newest observation at or before the issue instant.

Provenance

At issue the chain held 1,134,150 records, head 50ebf1b545d6f714c8c795bf5bf75c142533dfb30e8543a3f4188f0b326c8583. A single appending host; every record is ed25519-signed and carries its predecessor's hash.

The chain head is published daily and saved at the Internet Archive, so a reader can confirm the chain existed in this state on that date without taking our word for it. This is the anchor nearest this specimen; it is not this specimen's own head.

Nearest anchor: 2026-11-03T12:43:09Z, head 3958eb215031463242c8b9d84ee6a097aaf51e8963b99229aedff63ecf36826a at 1,092,600 records.

What was excluded, and why

1,340 hosts were excluded as stale. Last observed outside the window. Their verdicts are real and remain in the chain; they are simply not evidence about today, so they are not averaged into a rate presented as current.

Those hosts' own rate is 26.72%, and it must not be read against the figure above. This rate and the cross-section rate are over DISJOINT host sets, not the same hosts at two dates, so their difference is a sampling difference and must not be read as change over time. The series above is the only object in this document that measures change.

Pooling both sets gives 25.77% over 1,979 hosts. We do not quote that number. Every host ever probed live, whenever probed, in one rate: not restricted to the freshness window and therefore not a statement about today. Shown for comparison, NOT for quoting.

Recompute it

The estimator call, verbatim:

PYTHONPATH=. python3 - <<'EOF'
import pathlib
from phantom.curve import live_cross_section
print(live_cross_section(pathlib.Path('data'), now=1793782800.0))
EOF

Given the chain, this returns the figures above exactly. A later observation of the same host does not change them: the estimator is bounded at the issue instant, and tests/test_assay_reproducible.py grows a chain past that instant to prove it.

Limits, stated before you pay

What is never sold

A signed assay is one capability class across one declared population at one declared date, delivered with the method so the result can be recomputed against us. Scope is agreed in writing first. What an engagement costs, and on what terms, is set out on one page.

Request a scope