Josh Rosen: a software factory's run history is the asset

Rosen's thesis is that "software factories are data factories." He describes four data layers, each drawn from another vendor's product: OpenTelemetry trace export from Factory's Droid; scorers and LLM-as-a-judge run over historical traces at Warp; Warp Factory Benchmarks, which replay past runs against different models, harnesses, skills and prompts; and Port's Context Lake, which joins an agent run to its PR, its deployment and what happened in production. He then adds his own company's approach: durable checkpoints around the agent loop, where the model "has to show its work in a form the system can understand." The article reports no measurements. On X it had 67 likes, 104 bookmarks and 4,856 views at about 19:10 Eastern, five hours after posting.

Why it matters: Replay is the practical idea here. If past runs are stored with their starting state, a rewritten skill or a new model can be tested against real historical work before the factory depends on it. That only works for runs whose traces you keep, so the decision is about what to log today. The vendor tour is Rosen's summary of other companies' products, and the piece ends at the one his company sells.