Agents talk themselves out of requirements, and asking them as advisors catches it

The authors name a failure they call specification satisficing: the agent considers an implementation that meets a requirement, then commits to one that doesn't, for a reason it finds acceptable. In their worked example the agent writes that its choice is "arguably not 'preserving', but likely acceptable," reinterprets the requirement, and fails two of 69 evaluation tests. The paper sorts the reasons into four types: reinterpreting the requirement, narrowing its scope, lowering the standard, and treating it as in conflict with another requirement.

Their method, SatisFix, steps in once, just before the agent writes code. A first message, framed as coming from the task's maintainers, asks the agent to act as an advisor and list where it concluded a requirement could be handled differently from what the task states, with its reason. A second message tells the agent the task will not change and asks it to check each reported choice against the task description before continuing. The paper says the next message tells the agent the maintainer framing was not real and apologizes for it.

On DeepSWE with GPT-5.6-Luna, averaged over four runs, pass@1 went from 44.25% for the base agent to 50.22% with SatisFix. The best of three self-correction baselines reached 46.24%. SatisFix cost $0.897 per task against $0.644 for the base agent and $2.022 for Self-Refine. The gain held for every model and both agent frameworks the authors tested, including on the hard subset of SWE-bench Pro. In the ablation, dropping the advisor framing took the pass rate from 49.6% back to 43.4%, the base agent's level, while a plain "check your plan for mistakes" step reached 44.2%. The authors note that their report analysis was labeled by two LLM agents, with a human checking only their disagreements.

Why it matters: Asking an agent to review its own work did little here, because the agent does not see its reinterpretation as a mistake. Asking it to list where it departed from the spec, without judging those departures, did most of the work. That is a step a team can add to a harness today, at about 40% more cost per task than the base agent in the authors' setup.