← The Factory

Orchestration, State, Concurrency & Recovery

Separate durable session and state from inference and execution, and make recovery explicit.

high confidence

Confidence: high. Evidence: production telemetry and case study. Last substantive change: 2026-07.

This subsystem owns the machinery that keeps many agents working without stepping on each other: where state lives, how work is isolated, and what happens when something crashes mid-run.

The conclusion

Separate durable session and state from model inference and from execution, and use append-only traces, leases and idempotency, checkpoints, isolated work, bounded retries, and explicit recovery semantics. The brain, the hands, and the session are separable layers, and treating them separately is what makes concurrent, resumable work safe.

How the thinking got here

Chat sessions gave way to persistent agent loops, then to orchestrated worktrees, then to a decoupled model of brain, hands, and session with formalized execution traces. Large parallel runs surfaced the physical realities: worktree isolation prevents agents from clobbering each other, and the input-output bottleneck, not the model, becomes the limit.

Credible alternatives, and when each is right

Approach Right when
Stateless task runs short, independent work
Long-lived conversation continuity matters more than isolation
Event sourcing replay and audit are required
Workflow engine complex, long-running processes
Blackboard or actor model many collaborating agents
Isolated worktrees concurrent edits to one repository

Where it fails and what we still don't know

Failures include crash inconsistency, concurrent semantic conflicts that merge cleanly but break behavior, partial failure, and runaway loops. Evidence strongly converges on externalized state and isolated work; there is less evidence on which distributed architecture is best. Open questions include exactly-once illusions, replay correctness, and cross-organization coordination.

What would change our mind

A dominant, well-evidenced architecture for multi-agent state and recovery would replace today's convergent-but-varied practices with a standard.

Evidence and further reading