TDD inside the agent loop doesn't pay for itself, a small controlled study finds

Böckeler had Sonnet 4.6 build the same small, medium and large greenfield tasks with and without TDD instructions, twice each across five batches, then had Opus 4.8 blind-judge the resulting code and tests without knowing which workflow produced which. Across the small and medium tasks, Opus ranked the two non-TDD solutions first and second in most batches; TDD-instructed runs used 3 to 8.5x the tokens (by task size, per her appendix tables) with no consistent gain in mutation score, coverage, or design quality. Her reading of the transcripts: TDD's step-by-step structure suppresses the upfront design work that non-TDD runs did implicitly, so the design ends up locked to whatever shape the first test happened to take. She writes: "I personally have stopped telling my coding agents to write tests first, let alone do TDD... until I see evals or other strong arguments that convince me otherwise." The session data behind the numbers is public in her GitHub repo, birgitta410/tdd-comparisons.

Why it matters: telling an agent to follow TDD is common advice in harness and spec design, and this is the first quantified counter-evidence against it from a named practitioner with public data. Anyone enforcing TDD in an agent loop for quality reasons should read this before assuming it's paying for its token cost.