Ilya Razenshteyn: interviews with about two dozen engineers find stock tools and human review

Razenshteyn interviewed what he calls a couple of dozen engineers and researchers, from frontier labs to finance to small start-ups, about how they use coding agents at work. He dropped a planned survey because the stories were too varied, and he calls the result "qualitative and somewhat vibes-based." He does not say how interviewees were chosen.

He reports that almost everyone uses Claude or GPT models, and that harnesses are "almost exclusively" stock Claude Code or Codex. No one described swarms of agents, software factories or Gas Town-style setups; he gives cost and a lack of payoff as the reasons. The usual loop is plan, review the plan, implement, review the pull request with an agent and sometimes by hand, then merge. Several interviewees use one model to review another's work.

On correctness, he says code can go unread only for demos and one-off scripts, for strictly testable code such as a CUDA kernel, and for research code that only needs to be directionally correct. Elsewhere interviewees still read code or tests, and almost universally have some human peer review. The two failure modes he lists are messy code that accumulates as debt, and an agent misreading a requirement while the tests it writes reinforce the misreading. He also reports "slop" pull requests and design documents, and internal tools built in a day and then abandoned, as the main team-level problems.

Why it matters: This is self-reported and unmeasured, so it does not show what most teams do. It does give a dated counterpoint to factory-style setups from people working inside companies, and its practical rule is one a team can adopt now: mark which parts of a repository are reviewed and which are not.