A week of GitHub Copilot traces puts a number on what a model switch costs

Anonymized telemetry from one week of June 2026: 13.5M sessions, 3.2M users, 760.5M LLM calls, 774.7M tool calls, 27 models and 45 tools. Within a turn, prefix caching works, with a median hit rate of 98 percent. Three events break it. Crossing a turn boundary costs 26 percent of the hit rate on average, because the serving system evicts during user idle time. Switching models costs 67 percent, leaving an average hit rate of 8 percent, and switches are mostly reactive, triggered by errors or rate limiting. Context compaction fires in 7.8 percent of sessions, but those sessions carry 44.2 percent of all tokens; the median compaction drops 72.8 percent of prompt tokens, costs 66.1 percent of the cache hit rate, and eats 22 percent of the turn's wall-clock time. Tool failures occur in about 9 percent of turns and trigger retry loops the paper measures at up to 4x the compute, with failed run_build calls injecting seven to eight times more tokens than successful ones.

Why it matters: Two of the three cache-destroying events are things a practitioner chooses. Pinning a session to one model and keeping context below the compaction threshold are now decisions with published price tags, not preferences. The 4x retry amplification also says something uncomfortable about verification loops: an unreliable tool adds latency and multiplies compute without asking.