An open-weight model failed its tool-call schema on 2 calls in 25 until the server enforced it

Ubicloud, which serves Kimi and DeepSeek models, scored its deployments on Moonshot AI's K2VV ToolCall benchmark, a 2,000-request test of whether generated tool calls match the tool's schema. Serving Kimi K2.6 on vLLM 0.28 returned the expected schema 92% of the time, which the authors put as 2 failed calls in every 25. They had first carried constrained decoding in their own vLLM fork. When they moved to the upstream feature, their score fell back to 91%, because vLLM had changed the default for requests that do not explicitly turn it on. They report that vLLM maintainers saw problems with several models when it was on by default, including grammar translation bugs and a clash with speculative decoding, so the engine later added a --tool-strict-level flag. With constrained decoding enabled, they say they verified 100% schema accuracy across all the model versions they serve, and they name Kimi K2.6 and DeepSeek V4 Flash as the models where they had seen quality degrade.

The authors' own summary of the cause: "Change the inference engine and its settings, and you change the model's output quality."

The evidence is one provider's account on one benchmark. The post gives no run counts, no confidence intervals and no downstream task results, so the link from malformed calls to worse agent output is argued, not measured. It also reports that users of gpt-oss-20b, Qwen 3.6 and GLM 5.1 hit problems after the feature was enabled by default, so the fix is not safe to switch on blindly for every model.

Why it matters: If you run a coding agent against a self-hosted or third-party open-weight endpoint, malformed tool calls are a server-side cost you may be paying without seeing it, as error messages that pollute context and retries that add spend. The check is cheap: run the model's published tool-call benchmark against your own endpoint, and test the strict-tool-call setting per model before you rely on it.