Funnel: the model with faster calls finished its tasks 14% later

Funnel randomly assigned each new task in Reef, its internal data agent, to either GLM-5.2 (open weights) or Claude Opus 4.8. GLM averaged 7.8 seconds per model call against 18.1 for Opus. End to end it made 66% more tool calls (19 against 11 on average) and took 14% longer to finish. In two runs it executed the same successful SQL query 113 and 254 times. In one case the tool exposed only a 20-row preview, and the model kept re-running the query instead of noticing the limit. Funnel says it saw similar behavior from Kimi K2.6 and Qwen3.8 27B, with Qwen about twice as aggressive in tool use as GLM.

Funnel does not put the blame on the model alone. Some saved agent instructions named Athena tables and fields that did not exist, tool contracts were ambiguous, and nothing stopped repeated calls. In its words, "A successful tool call is not necessarily a useful tool call." The post gives no task count or variance, so the figures are one company's averages, and Funnel says it now runs GLM-5.3.

Why it matters: Per-call latency would have ranked these two models the wrong way round. If a team swaps models in a harness, the checks that matter are repeated-call detection, an execution budget and tool results that say whether they are complete, because a weaker model will find the gaps a stronger one papered over.