A harness that grows its control code from failure traces

The paper starts from a scaffold with fixed model and tool interfaces and no task-solving controller. Each round, an optimizer repairs a window of failed tasks at once, and it may edit only the entry function and the functions that appear in those failures' execution traces, within an edit budget. A held-out gate rolls back any sequence of repairs that lowers gate success, restoring the code along with the task cursor and counters. The authors direct parsing, validation, error recovery and stopping into code and keep model calls for semantic reasoning. Against a tool-calling agent on BrowseComp-Plus and WebArena-Verified, with deployment models from 4B to 120B parameters, the abstract reports 76.0 to 91.8% fewer LLM calls and 74.4 to 98.6% lower inference cost. It reports the highest mean success in five of six benchmark and model settings, and a mean 0.7 points behind the best in the sixth. We did not review the full results tables; these figures are the authors' summary in the abstract. Each benchmark's final evaluation uses 50 tasks.

Why it matters: For a factory that handles many tasks from one family, the paper's design gives a concrete rule for automated harness edits: scope each change to the code a failure touched, and accept it only if a held-out set does not regress. Both benchmarks are web research and browsing, so whether the approach carries over to coding tasks is untested here.