A refactor across 189 files with no human code review, and the program was not run until the 31st audit pass

The task was to break a structural guarantee the surrounding code was built on: that a UI panel stays open for the duration of an AI request. The target behaviour, a streaming generation that survives its panel closing and reattaches to the same live stream on reopening, did not exist before the change, so there was no test suite to satisfy. Abenhaïm's protocol replaces the missing oracle with a written specification. The agent produced one, then re-audited it against the real source code fourteen times, producing roughly 85 corrections and expanding scope from 110 to 160 affected files as dependencies surfaced. Cycle 14 returned nothing, so the cycle 13 specification was frozen and became the fixed reference for everything after. Code was generated against it, then audited against that same frozen document seventeen more times, producing 116 further corrections. The stopping rule was empirical: two consecutive verification passes returning zero findings, met at cycles 16 and 17.

Across 31 audit passes, 201 defects were corrected before any human executed the program, and the program was not run once during the whole phase; first manual execution came after the seventeenth verification cycle. The two commits total 288 changed files, 34,770 insertions and 16,422 deletions. Elapsed time three days, inference cost USD 2,430, agent implementation time 2 hours 21 minutes. The agent refused the first implementation request as non-atomic, on its own initiative: "A partial implementation would have left the repository in an inconsistent state and violated the specification's atomicity requirement." The harness doing the work runs under a system prompt of roughly 250,000 characters, accumulated rule by rule from observed failures.

The paper states three of its four caveats. The author designed and sells the agent under study, and declares the competing interest. The result is self-reported: he built the tool, ran the operation, and wrote it up, with 1,500+ pages of raw session logs published as evidence but the reporting itself not independent. It is one task on one closed-source codebase with one operator, no control condition, and no reproducibility claim. And a detail the abstract omits: a platform-side migration issue lost the correction lists for some refinement cycles, which are recorded in the paper as an estimate of five each, so "approximately 85" specification corrections is partly estimated rather than counted. The delivered behaviour is separately checkable, since the build ships publicly as version 2.3.0 and closing a panel mid-query stops killing the query at that version.

Why it matters: A frozen referent supplies an alternative check when human review is removed from agent output. That differs from "review it with a second agent," and the paper is explicit that it diverges from the second-reviewer guidance on exactly this point: the separation lives in the document, not in the identity of the reviewer. Treat it as one instrumented existence proof from an interested party, not as a method with a known success rate. It also provides a cheap stopping rule: audit until two consecutive passes find nothing, and count what those passes found.