Confidence: medium. Evidence: production telemetry, controlled study, and practitioner counterevidence. Last substantive change: 2026-08.
A finished change reaches the mainline only after a review gate decides who must inspect it and what evidence is enough to merge.
The conclusion
Review policy should be risk-calibrated. Low-risk changes can be auto-promoted when the evidence is strong, while critical changes need accountable human ownership even when agents perform the technical review. A controlled sabotage study found human review less reliable than teams often assume, so it cannot be the sole detector. At the same time, independent practitioner accounts from Horthy and Willison reject the literal no-read position for production code. Spend human attention where consequence, semantic judgment, and retained system understanding matter.
How the thinking got here
The starting point was a human reading every diff. Then agents began to pre-review, then automated low-risk review ran at scale, and then context-separated adversarial agent review with risk-calibrated merge rules emerged. Production systems reviewing hundreds of thousands of diffs showed that risk calibration, not universal human review, is what scales. The newer correction is that scalable detection and human comprehension are different jobs: automated evidence can carry the former while consequential domains still require the latter.
Credible alternatives, and when each is right
| Approach | Right when |
|---|---|
| Mandatory human review | high-stakes or regulated changes |
| Agent-only review | low-risk, high-volume changes with strong evidence |
| Sampled human audit | maintaining trust in an automated lane |
| Risk-based review | changes span very different blast radii |
| Pairwise or ensemble reviewers | independent perspectives reduce correlated error |
Where it fails and what we still don't know
Failures include correlated reviewer error, collusion between a builder and its reviewer, merge-queue backpressure when generation outruns review, and accountable owners who no longer understand what they operate. Evidence is strong for automated low-risk review and clear that human oversight is less reliable than assumed as a primary detector. Evidence is weaker on how much manual code reading preserves comprehension and incident readiness. Open questions include no-review merge outcomes, reviewer independence, change attribution, comprehension thresholds, and safe emergency bypasses.
What would change our mind
Controlled evidence comparing agent-only and human-gated review on the same consequential changes would settle how far the automated lane can safely extend.