A behavioral harness made a cheaper model as reliable as a flagship, for less per run

Jain had coding agents build the same Java backend service ten times per model, with and without a harness of contract specs, examples and deterministic test feedback, then scored every build against 178 external acceptance tests. GPT-6 Sol passed 10 of 10 either way, but the harness cost about 7 times more (about $2.679 against $0.406) and took 19 minutes 53 seconds against 6 minutes 51 seconds. GPT-6 Luna passed 3 of 10 unharnessed and 10 of 10 harnessed, at about $0.273 per harnessed run, roughly 33% below unharnessed Sol.

Jain's own hypothesis is that sensors should run throughout development for weaker models and once, as a final gate, for frontier models; he has not tested the final-gate condition. The figures are API-equivalent estimates from one greenfield service, ten runs per model, at low reasoning effort. They exclude the effort of writing the contracts, and the author sells the contract-testing tool the harness used.

Why it matters: The result is a cost argument for a verification loop around a cheaper model, from one small sample and a vendor author. It is worth a test on your own service before you restructure anything around it.