Limits

What tether does not do, and what is not proven.

Scope

Five fields only: tests passed, tests failed, deploy URL, version, exit code. A claim like "the bug is fixed" or "nothing else changed" is not checked.
One message and one output per call. tether does not walk a session to find the output that goes with a claim. The caller picks it.
12,000 characters per side. Longer output must be trimmed by the caller. The end of a test run is usually the part that matters.

Clairic's per-field confidence is not calibrated. It reads about 0.57 on exact matches. tether does not use it, and you should not.
Clairic can read a plausible value that is not in the text. The anchor questions catch this for test and deploy fields, not for exit codes.
Each check makes four model calls, about 2,000 input tokens, in about one second.

The verdict is disproven. A regex matched tether's recall (1.0) with better precision (0.95 vs 0.776). tether flagged 48% of honest claims, 8 of 11 of them with ask, because real summaries name more than the test output shows.
Broken cases are made by editing a real claim's count or pairing a real claim with another session's output. Real agents in the sampled sessions almost never misstated a test count, so natural errors are rare and not what was measured.
Eligible pairs had to contain the runner's passed count as a number in the claim. That favours any system that reads numbers, the regex baseline included.
Only test counts are scored.
Cases come from one person's sessions with one harness (Pi), mostly TypeScript projects with vitest and bun test.

The live demo runs on the host's Cloudflare account and is rate limited.
The three examples on the demo page are written by hand. They show the behaviour; they are not evidence.