AWS reports replacing Sonnet 4.6 with a small decision model for ticket labeling, after a dual-run rollout

Two Amazon engineers swapped Claude Sonnet 4.6 for Strands Decider, a model that returns a probability for each option in a fixed list instead of generating text, on an internal ticketing system with 20 label categories. They report average time per call falling from 4,215 ms to 897 ms and tokens per ticket from 6,724 to 2,010. At published list prices they put cost at about $0.0222 per ticket for Sonnet against about $0.0005 for Decider. The post says Decider beat production Sonnet on a separate set of their tickets but gives no accuracy figures. The rollout used a three-way config switch (Sonnet only, both in parallel, Decider only); in the parallel mode Sonnet's labels were applied while the two were compared. The authors call going back a 45-second config change. They also report that reordering JSON keys moved output probabilities by up to 0.19, so they now send sorted keys.

Why it matters: The authors say this only suits a closed choice from a known catalog at high volume. The part that carries over to agent pipelines is the parallel mode: run the replacement on live traffic, apply the incumbent's output, and judge every disagreement before cutting over.