← The Factory

Implementation & transformation

Where agents are already strong, and where they are not.

high confidence
Machine-readable Download Markdown

Confidence: high (within bounds). Evidence: case studies and benchmark. Last substantive change: 2026-08.

This is where plans become code. Agents write new systems, change existing ones, review patches, and convert codebases, with more public evidence here than in any other factory area.

The conclusion

Agents are already capable of substantial greenfield implementation, maintenance, review, and codebase conversion when the work has strong oracles and controllable seams. They are weaker on ambiguous product behavior and safety-critical novelty, where there is no oracle to check against.

How the thinking got here

Bounded patch benchmarks gave way to multi-file features and then to large-scale products, review systems, open-source maintenance pipelines, and million-line migrations. The largest published runtime migration used a frozen, language-independent conformance suite as its oracle, with zero tests skipped or deleted. That is a stronger behavioral constraint than builder-owned checks and is harder, but not impossible, to game. A separate practitioner account names this broader pattern conformance-driven development, including deriving behavioral tests from multiple existing implementations before building another.

Credible alternatives, and when each is right

Approach Right when
Human implementation with AI assistance novel or safety-critical work
Agent implementation with human review moderate risk, reviewable diffs
Agent implementer plus independent agent reviewer scale with context-separated checks
Regeneration rather than migration the spec is clearer than the legacy code
Mixed criticality lanes different risk tiers in one system

Where it fails and what we still don't know

The strongest results come from greenfield project setup, bounded changes, review, and conversions with mature test suites. The weakest come from ambiguous product behavior and safety-critical novelty. Open questions include net-new complex systems, hardware and embedded work, performance engineering, data-heavy systems, and maintenance beyond the first year.

What would change our mind

Sustained evidence of agents building novel, safety-critical systems without a pre-existing oracle would extend the boundary well past where it sits today.

Evidence and further reading