Cockroach Labs reports five months of agents working behind review gates

The authors describe agents that take issues through a workup, a plan and a review before any code is written, from April 21 to September 11. Their summary rule is "No code before a plan, no plan without review." After the first week they turned on a human approval mode that requires a person to look at every merge. They say the pipeline handled "over a million lines of code with only 7 total reverts," that they can reconstruct why any of 1,238 pull requests was merged, and that the run cost just over $135,000 in Claude tokens, about $84 for the average patient, their term for an issue. The Migration Assistant, now in preview for Postgres, was in their words almost entirely written this way. In an earlier Db2 test, they write, "we weren't reviewing any of the code that the hospital wrote during that initial test," and they list confirming its correctness as ongoing work.

The process also got heavy. A July audit of 25 skill files, a little over 100,000 words, flagged about 23,000 words as removable without changing any gate, command or template. One fix that came to a single line took eleven rework rounds over two days. They now hand a pull request to an Attending reviewer after four consecutive rework rounds that each turned up a new blocking finding, or after six substantive rounds of any kind.

Why it matters: This is a company's account of its own repository, so the totals are its figures and nothing here tests them independently. What a team can borrow is concrete: a plan-review gate before code, human approval on every merge, and numeric thresholds that stop a review loop from running indefinitely. Whether those thresholds work as intended is not shown yet.