Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix · Xiaonan Xu (Georgia Institute of Technology) and Wenjing Wu (University of Colorado Boulder) · arXiv, October 22, 2026 (preprint, submitted to SN Computer Science)
Agents follow a stale CLAUDE.md or AGENTS.md over the code, and one sentence reverses it
Machine-readable
Download Markdown
The authors built 40 small Python projects in which the instruction file asks for one convention and the existing code uses another, such as print instead of logging. Claude Code (with Claude Sonnet 5.5) and Codex CLI (with GPT-6 Sol) each added a feature to every project, with and without a sentence in the user-level instruction file, for 320 runs. Without the sentence, 90.1% of the agents' choices followed the stale file and 4.2% followed the code. Claude Code named the conflict in its final reply in 79 of 80 runs and still followed the file. Codex CLI rewrote existing code to match the file in 20 of 80 runs, and the project's original tests then failed in 17 of those.
The added sentence, placed in the user-level file, tells the agent to follow the existing code when documentation and code disagree and to list each inconsistency it finds in its final reply. With it, 91.4% of choices followed the code, no Codex CLI run rewrote existing code, and tests passed in all 160 runs, up from 89.4% without it. The authors attribute the behavior to the file reaching the model as an instruction and the code reaching it as tool output, which models are trained to rank lower. The projects, run records and analysis scripts are listed as available in a public repository; we have not rerun them.
Why it matters: If these results carry over to real repositories, an old line in CLAUDE.md or AGENTS.md can cost a team more than a missing one, and agents will not warn you before acting on it. The study covers only synthetic Python projects, two agents and single unattended feature tasks, so test the sentence on your own repository before relying on it.