← In the News

A preprint proposes boundary criteria for the layer above the agent harness

Beyond the Agent Harness: Boundary Criteria for Software-Factory Orchestration, A Single-Operator Case Study · Lei Hua, independent researcher · arXiv cs.SE, submitted November 3, 2026

Machine-readable Download Markdown

Hua does not claim the factory layer is new. The paper says "The existence of such a layer is not our contribution," and asks instead for a rule to decide which layer a given mechanism or failure belongs to. The rule is built on a definition of a run: one agent session from dispatch to terminal status. A mechanism confined to one run belongs to the agent harness. One that coordinates runs, passes state to a later run, or is updated from run outcomes belongs to the factory harness. One that must change with the application domain belongs to a domain layer. Two residual tests assign infrastructure and model causes.

For failures, the paper codes three fields separately: where the defect originated, which control failed to stop it, and where it was fixed. A missing gate counts as a control gap and does not move the origin, so a defect that a factory gate could have caught is not automatically a factory failure.

The checks are modest, and the author says so. Two model coders of the same family applied the frozen criteria to 46 held-out units. They agreed on all 26 mechanism layers and on 18 to 19 of 20 incident fields (kappa 0.86 to 0.93). Both still recorded the incident as undetermined or unrecorded in 8 to 13 of 20 cases per field, and each flagged half of all units as ambiguous. A revised version that separates mechanism classification from incident attribution agreed less (kappa 0.78 to 0.93) and has not been validated on new material. The abstract states that "Agreement here is neither accuracy nor usefulness, and human coders have not yet been tested." The case system serves six projects and was designed, operated and evaluated by the same person, with small samples and unblinded adjudication. The paper lists seven hypotheses as preliminary.

We read the abstract and the framework and case-setup sections. We did not read the evaluation sections or the appendices, and we did not open the released data files.

Why it matters: If you run agents above a single harness, the failure coding is the part to borrow first. Recording origin, missing control and fix location as separate answers keeps a gate you forgot to build from being blamed as the cause of the defect. Treat the layer rules as a proposal to test against your own incident history, since the only validation so far is machine coders working from written descriptions.