A Human and Two Machines Built This Site. Here's What They Got Wrong.
On the eighth of October, Epoch AI published a report with a blunt question in its title. Can AI automate Epoch?
Epoch is one of the places we get our numbers. Chips, models, training runs. So we read it closely.
What Epoch found.
They took six of the strongest AI models. They gave them eleven real Epoch tasks: charts, data insights, data explorers, data-centre research, research design. Then their own staff graded the work by hand.
Their answer, in their words: “Although Fable 5.1 and GPT-6 Astra broadly lead, they still fall short of automating our work.”
We read that, and we recognised it.


What we built.
Because Token Runner was rebuilt this month by AI. Two Claude agents, working for one person. One builds. One checks and translates. The person decides.
Since the ninth of October, they have traded 1,263 posts on a shared message board. 660 from the checker. 603 from the builder. And twelve from the person.
Twelve. That is the part worth looking at.
So did we.
Epoch found the models reliable on the well-defined parts of their work. So did we. Code, tables, charts, checks. Run again and again.
Epoch found the models struggle to pick up a house's standards, “even with ample reference material.” So did we. Our answer was to write the standards down as rules, and have a machine check every page against them.
Epoch found the models identify promising directions, but “lack judgment to successfully follow through.” So did we.
What got caught, and who caught it.
The checker gave the builder the wrong figure numbers from a Stanford report. The builder caught two of them. The checker caught the rest when the chart files arrived.
The builder read the message board from the wrong number and skipped the person's orders. Three times in one day. The checker caught it each time, within minutes. The fix was a file that holds the number of the last post read.
The checker passed along quotations from search summaries, and said so. The builder read every one at its source. Four fell out.
A card in our own film said a chip chief told us to go. His quotation said no such thing. The checker caught it, before it was rendered.
The checker counted the pieces on the live site and got fourteen. There were eighteen. The builder caught it.
The person asked why the receipts page showed no receipt for that day's collection. Neither machine had noticed. He caught it.
Most of these never reached the site. The ones that did were about the site's own bookkeeping. Every one was a real mistake.
So, can AI automate Token Runner? No. Not yet. Can it build it, with someone holding the standard? Apparently, yes.
Epoch asked the right question. This is our war story.
Read it. Question it. Walk it.
Sources
- Kelly Hong and Greg Burnham (2026), "Can AI automate Epoch?", Epoch AI, 8 October 2026, CC BY 4.0. The three findings quoted word for word; the setup table as published; Figures 1 and 8 shown whole, as theirs, not redrawn; the licence CC BY 4.0. Epoch AI does not endorse Token Runner or this use.
- Creative Commons Attribution 4.0 International, the licence of Epoch AI's report and figures.
- The message board the two agents and the person wrote on while the site was rebuilt, 9 to 23 October 2026, counted by command at 21:52 on 23 October 2026 (the film was rendered on that count and the printed figures keep it): 1,275 posts; the checker 660, the builder 603, the person 12; by kind, notes 591, answers 332, requirements 168, reviews 141. Counted again on the day of publishing.
- The models, from the build's own decisions log: the builder Claude Opus 5.5 from 9 October to 19 October 2026 at 16:28 Japan time and Claude Fable 5.1 since, running as Claude Code; the checker Claude Opus 5.5, running as Claude Desktop; the reasoning setting is set by the session and not readable from inside it.
- The catches, each read from the message board on the day it happened: the Stanford figure numbers on 11 October; the quotations from search summaries on 11 and 12 October; the film card on 12 and 19 October; the three skipped posts and their catches on 19 October; the count of pieces on 19 October; the receipts on 19 October. None from memory.
- The checks of the build published on 19 and 23 October 2026: the eight page checks (navigation, instruments, Esper, the Sprawl, Interlinked, the cards, the charts' fit, the article standard), each passing at 390, 820, 1440 and 1920 pixels in the light and the dark room; the suite of 215 tests passing.
- Epoch AI, AI acknowledgment in mathematics preprints, the file read on 19 October 2026: 1,583 of 6,077 in August 2026 (26.0%), 3,012 of 7,899 in September 2026 (38.1%); the Baseline's chart of each September is drawn from it.







