ACM Queue puts the evidence behind the case against measuring agents by lines of code

A 2025 study of more than 450 Microsoft engineers found developers spend 14% of their time writing code. The authors take it to its conclusion. If coding is about 15% of the work, an assist that makes coding twice as fast raises overall productivity by under 15%, and the other 85% is untouched. On measurement they are blunt: lines of code, including AI-generated lines of code, are "neither statistically valid nor meaningful indicators of impact," a finding they trace to a 2014 statistical study. On adoption they report that 80% of developers use these tools while only 29% trust their accuracy. On effect size they set a 55% productivity gain from one study against a 2025 finding that experienced open-source developers took 18% longer with AI tools.

The article is about coding assistants rather than autonomous agents, and its statistics are its citation of other people's studies rather than new measurement by these authors. What survives the translation is one sentence: an assistant "tackles the 'inner loop' of writing code in the IDE but leaves the 'outer loop' of development largely unchanged," and accelerating generation "can simply move the pressure downstream" into review, testing and integration.

Why it matters: The evidence supports a position the corpus already holds on other grounds, that harness effort belongs in verification rather than generation. It also warns against a common dashboard measure. If you are reporting agent impact as AI-authored lines, the authors' position is that you are reporting a number that does not measure the thing you care about, and that this was established a decade before agents existed.