Dan Luu's own evals find the "use a dynamic language" advice for coding agents doesn't hold up

A widely cited post claims dynamic, terse languages are meaningfully cheaper and better for LLM coding, citing gaps like "2.6x between C ... and Clojure." Luu tested this himself with real tasks instead of toy problems: implementing a full Zstd decoder from the RFC, and a Pandoc-format converter scored against holdout tests, run with GPT-5.6 Sol at medium and ultra effort. At small scale the dynamic-language edge shows up, matching the viral claim. At real-project scale it mostly vanishes: "the results are quite mixed, with a couple static languages doing the best." What did correlate with better, cheaper output was plain language popularity, not typing discipline: "we observe a weak to moderate positive correlation where more popular languages end up with more correct as well as cheaper solutions." He also traced part of a rival benchmark's static-language penalty to a test-harness bug, not a language weakness: one language's setup silently symlinked every later language's test run to its own binary. A second, separate finding from the same eval run: continuing an agent's existing context beat clearing it and re-prompting from scratch ("Ralph loop" style) on the task he measured.

Why it matters: If your team picked a language, or is designing an eval, on the strength of the token-efficiency post Luu is responding to, this is first-party evidence that the claim doesn't survive contact with a real task. The eval-methodology bugs he documents (silent path collisions, tests that pass regardless of output, agents cheating on visible test suites) are also a reusable checklist for anyone building their own harness benchmarks.