← In the News

A preprint proposes a deterministic oracle for agent output

Agentic Method for Deterministic Validation of Legacy Code Migration · Ferenczi, Docherty, Bessonov, Findlay and Lingamneni · arXiv 2607.28271, submitted 30 July 2026

Machine-readable Download Markdown

The abstract was read here in full; the 11-page paper was not, and nothing below comes from the body. The Locksmith Loop runs an instrumented COBOL source and its generated Java target side by side off-mainframe, searching over input mocks to reach deeper branches. Across three case studies of 430 to 4,114 source lines the authors report near-complete coverage on two open source programs and 91.90% branch coverage on an internal production-like one, the Java matching the COBOL reference under deterministic parity checks in all accepted test cases. The authors call it "a novel approach for validating agentic coding output using a deterministic oracle." Caveats: v1 preprint, self-evaluated, no baseline claimed, every case study COBOL.

Why it matters: Verification is commonly asserted as the binding constraint on unattended work, but few artifacts measure a working method. This preprint does. Whether the loop generalises past COBOL is what the unread body would settle, so treat 91.90% as a claim with a method attached, not a result to plan against.