Confidence: medium. Evidence: case study and production telemetry. Last substantive change: 2026-07.
This subsystem owns what happens after software is running: watching it, noticing when it breaks, diagnosing why, and either fixing it or handing off to a human.
The conclusion
The factory is incomplete if it stops at merge. It needs outcome telemetry, user signals, anomaly detection, diagnosis, repair, incident records, and a path to escalate rather than fabricate success. The most dangerous operational failure is the one where the system reports a confident, plausible, false success instead of surfacing the problem.
How the thinking got here
Continuous-integration logs gave way to production observability, then to healer loops that diagnose and repair, then to taxonomies of silent and fail-plausible failure and to governance that monitors the monitor. Every step strengthens the operator's ability to trust that green means green, which is exactly what fail-plausible failure destroys.
Agents make execution evidence more important because source code no longer predicts a single path through the system. Sydney Runkle puts it plainly: "the story is actually in traces for agents". A trace becomes useful evidence only when the system connects it to outcomes, feedback, and the version of the harness that produced it.
Credible alternatives, and when each is right
| Approach | Right when |
|---|---|
| Human SRE | high-stakes production, novel incidents |
| Agent-assisted diagnosis | speeding human incident response |
| Auto-remediation for known classes | well-understood, recurring failures |
| Full closed-loop healing | reversible, well-instrumented systems |
| Chaos-tested combinations | resilience must be proven, not assumed |
Where it fails and what we still don't know
Failures include false remediation, correlated errors between a monitor and its healer, and fabricated success that hides a real incident. Evidence is moderate; feasibility is shown, but longitudinal independent evidence is scarce. Open questions include incident command, customer communication, service-level ownership, and safe stop behavior.
What would change our mind
Longitudinal evidence that closed-loop healing improves real production outcomes over quarters, without accumulating hidden failure, would extend autonomy into operations.
Evidence and further reading
- The Art of Loop Engineering: How to Build Agents That Improve Over Time
- How Coding Agents Fail (20,574 real sessions)
- When Errors Become Narratives: a taxonomy of silent failures
- LLMs Corrupt Your Documents When You Delegate
- Coding Agents Do Not Know When to Act
- StrongDM: Software Factories and the Agentic Moment