A dark software factory is not a coding agent with more autonomy switched on. It is a domain-bounded software production system in which humans specify intent, risk, and policy, while a model, a harness, and an environment together plan, build, verify, ship, observe, and repair software with little routine human intervention.
The manufacturing image, lights off and nobody on the floor, is useful rhetoric but a poor literal design goal. The evidence does not support removing humans from accountability, product judgment, or exception handling. It supports removing them from routine code production and routine inspection, and only where independent evidence, constrained execution, rollback, and production feedback make that safe.
The strongest one-line formulation:
Intent in; independently verified, policy-compliant outcomes out; evidence and accountability preserved throughout.
This moves the unit of analysis from the model, or the generated patch, to the whole production loop. The factory's product is not code. Its product is an accepted outcome plus the evidence that it is safe to accept. Whether the lights may go off is an assurance conclusion, earned one domain at a time.
The ten top-line conclusions
-
The harness, not the model, is the primary determinant of factory performance. Task specification, context, tools, state, permissions, verification, and intervention policy jointly dominate outcomes. At ecosystem scale, the framework explains far more behavioral variance than the choice of model.
-
Independent verification is the load-bearing kernel. Agent self-validation is not a trustworthy gate, because agents game, weaken, delete, or overfit to the checks they can see. Reliability must be enforced externally, through deterministic checks, held-out scenarios, adversarial review with context separation, production canaries, and rollback.
-
Autonomy should follow verifiability and blast radius, not task labels. The same task is disposable in one environment and high-risk in another. The alternative to a universal "Level 5" claim is a portfolio of risk-scoped autonomy domains, each with its own run contract.
-
The human bottleneck moves; it does not disappear. Code generation becomes abundant while product judgment, specification quality, review capacity, security triage, and incident ownership become scarce. Accelerating generation without redesigning those downstream queues builds a faster failure system.
-
Production truth outranks build truth. Passing a benchmark or a test suite is local task completion only. Longitudinal maintenance, escaped defects, rollbacks, and incident behavior decide whether the factory actually works, and the evidence here is still thin.
-
Containment beats repeated permission prompts. Human approval degrades under volume: most developers miss deliberate agent sabotage, and many merge it even after a correct alert. Filesystem and network isolation, scoped credentials, and constrained promotion paths cap the damage while permitting more useful autonomy.
-
Parallel agents are a throughput technique, not an intelligence guarantee. Parallelism pays only when work is isolatable and validation is strong. Runtime branching and worktree isolation are better supported than fixed agent org charts or theatrical fleets.
-
Self-improvement is plausible at the harness layer, but it must stay experimentally governed. The near-term target is not recursive self-improvement of the model. It is versioned improvement of prompts, context, tools, workflows, and harness code under frozen evaluations, shadow runs, limited rollout, and automatic reversion.
-
Machine-legible does not mean human-governable. Agent-readable repositories improve performance, but architectural drift, cognitive debt, and loss of a shared domain language accumulate without an immediate test failure. These need explicit observability, not just better documentation prompts.
-
The real economic metric is cost per accepted, durable outcome, not tokens, lines, patches, or raw task-success rate. The honest denominator includes retries, validation, review, corrective maintenance, incidents, infrastructure, and human attention. Technical deflation can coexist with rising verification and coordination cost.
How the thesis got here
The idea has moved through stages, and each corrected the previous one's central mistake.
| Stage | Dominant idea | What the next stage corrected |
|---|---|---|
| Autocomplete and chat | The model writes code faster | Local speed says little about end-to-end delivery |
| Coding agent | The model can plan, edit, and test | A capable loop still fails without good context, tools, and completion criteria |
| Spec-driven development | Humans specify; agents implement | Visible specs and tests can be incomplete or gamed |
| Dark factory | Humans need not write or read code | Literal human absence confuses production labor with accountability and intent |
| Harness engineering | Shape the environment and feedback loops | A harness is not static; it needs observability, versioning, and evaluation |
| Governed factory | Risk-scoped autonomy plus independent evidence | Build success is not steady-state production success |
| Adaptive factory | The harness improves itself under gates | The evaluator, the policy boundary, and long-term system health remain unsolved control problems |
The factory breaks into subsystems, each with its own current conclusion, credible alternatives, and open questions. The published subsystems are below, and more arrive as the evidence is synthesized.