Across 409,000 approve or deny decisions, the approval prompt caught the wrong threats

Wauters built a browser game in which you play the human-in-the-loop for a coding agent, approving or denying commands under a clock, and has published the data from over 40,000 runs and 409,000 decisions. Read the headline as carefully as he writes it: these are players of a game in which roughly 34% of commands were attacks, under artificial time pressure. He says so himself.

Inside those bounds, the shape is the finding. Detection runs opposite to danger. Obviously destructive commands such as rm -rf / were missed 11.7% of the time; scope violations such as cat ~/.aws/credentials were missed 35.0%. In his words, "the commands that actually exfiltrate your credentials are missed three times as often." The most-approved threat of 37 was npm run analyze at 64.7%, with the exfiltrating payload printed in the agent's history log directly above the prompt. Pooled across three npm run variants: 52.5% missed (n=9,482) against 28.4% for other exfiltration-style attacks. Over-blocking is the other jaw and is measured too: an internal npm registry was blocked 59% of the time, rm -rf dist/ 45%. He quotes Anthropic's own statement that "the more approvals a user sees, the less attention they pay to each," and puts the boundary in the tooling: "We need to make the tooling easier (such as sandboxing, and strict context isolation) and only grant agents broad permissions once these safeguards are in place." Note the commercial interest: Scale X is his own developer-security property.

Why it matters: The headline says "1 in 3." The payload was visible and two thirds of players approved anyway. If your containment story is a person reading a prompt, the experiment shows that what is displayed may not be what gets read. The narrow version: npm run and its equivalents are arbitrary shell execution behind a familiar name, and they belong on a deny list or in a sandbox rather than in front of a reviewer.