A small model trained to rewrite a solver's harness beats its larger teacher on average, on reasoning tasks

The authors train a proposer model with reinforcement learning to edit the executable harness around a frozen solver, using execution reports as feedback. No weights change at test time; the harness revisions play that role. On 21 Reasoning Gym task families the proposer never saw, a 4B proposer's single revisions averaged 0.62 against 0.32 for the untrained base. They also scored above the 35B teacher's 0.56 on average, though the teacher did better under oracle best-of-eight selection.

In multi-hop question answering, a proposer trained on HotpotQA carried over to MuSiQue and 2WikiMultihopQA. On MuSiQue, independent revisions from the single-step proposer averaged 0.15 exact match, barely above the seed harness, against 0.27 after ten rounds of building on the current harness. The authors report that training on whole revision sequences, as opposed to single revisions, helped in some settings and not others. On Reasoning Gym the learned harnesses were mostly interpreter loops, and generated helper components contributed to the answer in only a minority of harnesses that integrated them. The paper names its limits: a fixed solver, prescribed revision prompts and one teacher.

Why it matters: The tasks are reasoning puzzles and retrieval QA, not coding agents, so nothing here shows that a learned proposer improves a software factory's harness. What it does show is that harness edits driven by execution feedback can transfer to tasks the proposer was not trained on, and that repeated revision beat independent samples on the harder QA benchmark. I did not find a code release.