Sarnacki reruns the Campfire rewrite benchmarks and finds the prompt left the hard choices to chance

Sarnacki critiques the Rust, Elixir and Go rewrites of DHH's Campfire Once, which he describes as largely agent-written. The versions differ for reasons unrelated to language: the Rust one drops the CSRF token and replaces Redis with in-process queues, while the Elixir one stays closer to the Rails original. "If your prompt is not specific enough, many decisions are a coin flip," he writes.

He then turns to the benchmarks. A closed-loop test by Zach Daniels found the Rust version delivered about 1% of new-post notifications under heavy load. Sarnacki says that kind of test measures throughput and hides how a system fails, so he reran at a constant 100 POSTs a second. He reports that Rust delivered about 14% of events and Elixir about 60%, but Elixir timed out on about 23% of POSTs, and as the post reads, worst-case delivery latency for Elixir came close to 180 seconds. Raising the Rust broadcast channel capacity from 256 to 16384 lifted delivery to about 90% and pushed maximum latency from 11 seconds to over 130. He argues that failing fast is the better behavior. He also reports 1.8GB of memory use in Elixir during the same test.

These figures come from his own reruns. The post links no harness, and the rewrites themselves were not independently checked.

Why it matters: An agent asked for a rewrite answers questions about compatibility, queue bounds and failure behavior whether or not the prompt raised them, and one throughput number does not show which answers it chose. Setting those constraints, and testing for them, is work the person writing the prompt still has to do. The numbers are one author's account.