---
title: 'In the News: August 4, 2026'
description: 'A controlled two-agent ablation finds AGENTS.md and CLAUDE.md do not measurably move coding-agent correctness on either Claude Code or Codex.'
canonical_url: 'https://darkfactory.dev/news/2026-08-04-morning'
markdown_url: 'https://darkfactory.dev/news/2026-08-04-morning.md'
collection: news
date_published: '2026-08-04T10:30:00-04:00'
date_modified: '2026-08-04T10:30:00-04:00'
---

# In the News: August 4, 2026


Two arXiv papers this cycle measure something the field has been doing on faith.
One finds the context file you spent an afternoon writing does not change whether
your agent's code passes. The other finds that when an edit requires deleting
code, models mostly wrap it in a guard instead, and the tests do not notice.

## 1. A controlled ablation finds AGENTS.md does not move correctness on either frontier agent

**[Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories](https://arxiv.org/abs/2607.27250)** · Prakhar Khatri, independent researcher · arXiv:2607.27250v1 [cs.SE], 28 July 2026

Khatri ran 291 agent runs, 288 of them evaluated, across Claude Code
(claude-sonnet-4-6) and Codex CLI (gpt-5.5), on 17 tasks mined from merged pull
requests in three Python repositories, scored against each PR's own hidden
tests. Three conditions: no context at all, the full AGENTS.md injected into the
system prompt every turn, and a retrievable wiki the agent reads on demand. Pass
rates barely separate. Claude: 53.3, 55.6, 55.6 percent. Codex: 58.8, 56.9, 52.9
percent. Equivalence testing bounds every pairwise difference under 10 points for
Claude and 15 for Codex. A pre-registered 36-cell probe re-ran the two
convention-closest near-misses on both agents, and the result is the paper's
sharpest line: "across both agents the real AGENTS.md never converts a near-miss
to a pass." The failure triage explains it. Near-misses fail on pattern choice
and exact wiring, not on repository facts a document could supply.

One thing did move. On opshin, the one repository whose AGENTS.md warns that the
test suite takes over 20 minutes, Claude's blind full-suite pytest runs fell from
3.67 per cell with no context to 2.44 to 1.67, and wall-clock time fell about 24
percent. Khatri reports that as exploratory and underpowered at n=5.

**Why it matters:** This is the first study to vary injection strategy across two
agent families under gold-test evaluation, and its reading is narrow but usable:
a generic style-and-conventions file buys you process, not correctness. The
conclusion is not to delete your CLAUDE.md. It is to stop expecting it to raise
your pass rate, and to put in it the things that change what the agent *does*,
like which test command is cheap. Khatri states his own caveats: 3 Python
repositories, a minimum detectable effect above 30 points, a selective arm whose
corpus is not content-matched, and two model snapshots.

## 2. Models systematically avoid deleting code, and the tests that pass them do not check

**[To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing](https://arxiv.org/abs/2607.28887)** · Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia, Gopi Krishnan Rajbahadur, Ahmed E. Hassan · arXiv:2607.28887v1 [cs.SE], 30 July 2026

**This item is written from the paper's abstract. The body is unread by this
edition.** On that basis: the authors name "deletion avoidance, the systematic
tendency to retain code that an intended edit requires removing." Across the five
leading models on the SWE-bench Verified leaderboard, deletion recall against the
developer patch reaches at most 71.7 percent even on tasks all five solve, and
models find the right file for over 92 percent of required deletions while
cutting the exact line in under 52 percent of cases. Instead, they report, "29.0%
of passing patches wrap the targeted code in a guard or fallback, a pattern we
call Guard-and-Go." Retrofitting 34 Verified tasks with tests that fail if the
targeted code remains drops four frontier models from 63.2 to 41.9 percent. A
new 200-task benchmark, CanItDelete, is mined from commits whose entire required
edit is a deletion; the best model still fails one in five.

**Why it matters:** If it holds up on a full read, this is a verification-loop
finding before it is a model finding. A green suite is being read as "the edit
was made" when it only establishes "the new behaviour works." Anyone running
agents against a test oracle has a concrete thing to add: an assertion that the
removed thing is gone. Affiliations are not on the abstract page and this edition
did not establish them.

## 3. Claude Code 2.1.221 closes two permission-check bypasses and adds credential masking

**[Claude Code changelog, version 2.1.221](https://code.claude.com/docs/en/changelog)** · Anthropic · first-party release notes, 4 August 2026

Today's release fixes, in Anthropic's own words, "a Bash tool permission-check
bypass where zsh could execute hidden commands in `[[ ]]` regex conditionals;
affected commands now prompt for permission," plus a PowerShell permission check
that mishandled paths containing quote characters on Windows. It also adds
`mode: "mask"` for sandbox credential files on Linux and WSL, where sandboxed
commands read a sentinel copy, the whole file or just the spans matched by an
`extract` regex, while the sandbox proxy substitutes the real value on egress.
On macOS, file masking falls back to `deny`.

**Why it matters:** If your containment story is "the harness prompts before it
runs anything I have not allowed," a class of shell constructs sat outside that
boundary until today. Upgrade and inspect whatever else your permission layer
pattern-matches on. The masking mode matters more in the long
run: it lets an agent use a credential without ever reading it, which is the
shape containment has to take before unattended runs touch anything real.

## 4. Goedecke: the human is the bottleneck, not the model

**[LLMs reward expertise](https://www.seangoedecke.com/llms-reward-expertise/)** · Sean Goedecke, staff engineer at GitHub · seangoedecke.com, 24 July 2026

Goedecke's claim is that the most important prompting skill is domain expertise
in the thing being prompted about, and his evidence is a specific artifact: a
[public ChatGPT transcript](https://chatgpt.com/share/6a5fdc7a-d6f8-83e8-bbea-8deb42cfed56)
of Terence Tao working on a counterexample to the Jacobian Conjecture. "This is
not the same ChatGPT I talk to," Goedecke writes. He reads the transcript for
technique: Tao's messages are short, he pushes back obliquely rather than
contradicting, and he almost never takes the model's suggestion about where to go
next. The conclusion is that "the human is the bottleneck, not the model." The
essay offers no measurement. It was the top item on Hacker News `/active` at
09:00 EDT, with 1,089 points and 454 comments at 15.0 hours.

**Why it matters:** It sits directly against the comprehension-debt material this
publication has been tracking all week, from the opposite side. If an agent's value
scales with how well you already know the system, then every practice that lets
you stop knowing it is spending the thing that makes the agent useful.

---

## Also this cycle

- **[Elon Musk: "Source code is on the verge of becoming like assembly"](https://x.com/elonmusk/status/2084304083851034949)** · X · Musk's post, dated 3 August, argues the next step is "getting rid of 'source code' entirely and just making an efficient binary directly with AI." Dex Horthy of HumanLayer, whose *Why Software Factories Fail* is the most-cited artifact in this lane, quote-tweeted it to disagree: "It is the future but we are far from being 'on the verge' of it for serious software with uptime." Horthy's post stood at 270 favourites and 33K views at roughly 18 hours. Its permalink was not captured, so it is reached here through Musk's post.

- **[Gordon Mickel claims a software factory that does not fail](https://x.com/gmickel/status/2084406133062533521)** · X · Mickel, who leads AI at GrowthFactors, says he has been running his own factory setup against SlopCodeBench and "they don't need to fail." A write-up has been promised twice and is not published. Reported as a claim by a named practitioner, not a result: no methodology, no numbers, no artifact yet. At 8 favourites and 8.4K views at 14 hours it is travelling far less well than Horthy's reply to it.