Snap describes an experiment loop where the harness is deterministic and the agent works only at the edges

Snap's ML engineers run an agent platform called Alchemist, built around a harness that validates plans, submits runs, polls results and enforces budgets as code the agent calls rather than behavior it improvises. The agent proposes hypotheses at the start and recommends next steps at the end. Per the post, each plan is approved by the sha256 fingerprint of its file, an edited plan counts as unapproved, and the harness never retries a failed step on its own.

For its first recipe, feature selection, Snap reports a drop from four weeks to two per cycle and 30% of model features removed with no regression in the study's evaluation metrics. The baseline was one manual study on a Story ads model: four weeks, 130 workflow submissions, one engineer. That is a single comparison reported by the team that built the system, and the post gives no independent measurement.

Why it matters: The design choice is the useful part. Snap puts the steps that must not vary into code and treats approval as a hash of the plan, which is a pattern a spec-and-verify pipeline can borrow without taking on Snap's numbers.