arXiv:2605.26128v1 Announce Type: new 
Abstract: Production LLM systems increasingly require machine-readable outputs: JSON objects, typed traces, regex-constrained fields, and tool-call schemas. This paper targets on-device and low-cost small language model (SLM) deployments, where sub-3B models are attractive for privacy, latency, and commodity hardware but have limited capacity to satisfy schemas while solving tasks. The usual engineering assumption is that hard output constraints improve reliability without changing the underlying answer. We show that this assumption is unsafe for small models. We introduce \emph{constraint tax}, a measurement protocol for isolating the answer and executable-accuracy loss caused by structured-output constraints at fixed model, fixed task distribution, and fixed problem instances. Across 15,000 commodity-GPU generations with Qwen2.5-0.5B, Qwen2.5-1.5B, and SmolLM2-1.7B, hard answer-only schema decoding raises schema validity from 61.5\% to 100.0\%, but lowers answer accuracy from 19.7\% to 11.0\% and increases wrong-valid-schema outputs from 49.5\% to 88.9\%. The strongest industry analogue is a deterministic calendar tool-call task: Qwen2.5-1.5B achieves 91.5\% executable accuracy with prompt-only JSON but only 48.0\% under the same hard tool-call schema, while both modes are 100.0\% schema-valid. The error is semantic, not structural. We also show that the 3B boundary still pays a direct-schema tax and that delayed packaging supports a constructive design pattern: reason free, constrain late. The practical conclusion is direct: production systems should report schema validity, answer accuracy, executable accuracy, and wrong-valid-schema rate separately.

قدمت دراسة حديثة مفهوم 'ضريبة القيود' لقياس التوازن بين الصلاحية والدقة في المخرجات المنظمة التي تنتجها نماذج اللغة الصغيرة (SLMs)، مع التركيز بشكل خاص على نماذج مثل Qwen2.5-0.5B وSmolLM2-1.7B. تبرز الأبحاث أن القيود الصارمة على المخرجات قد تؤدي إلى شعور زائف بالموثوقية، حيث قد لا تحسن الدقة الأساسية للإجابات المقدمة من هذه النماذج.

Un estudio reciente introdujo el concepto de 'impuesto de restricción' para medir los compromisos entre validez y corrección en las salidas estructuradas generadas por pequeños modelos de lenguaje (SLMs), centrándose particularmente en modelos como Qwen2.5-0.5B y SmolLM2-1.7B. La investigación destaca que las restricciones de salida estrictas pueden llevar a una falsa sensación de fiabilidad, ya que pueden no mejorar la precisión subyacente de las respuestas proporcionadas por estos modelos.

Une étude récente a introduit le concept de 'taxe de contrainte' pour mesurer les compromis entre validité et exactitude dans les sorties structurées générées par de petits modèles de langage (SLMs), en se concentrant particulièrement sur des modèles comme Qwen2.5-0.5B et SmolLM2-1.7B. La recherche souligne que des contraintes de sortie strictes peuvent engendrer un faux sentiment de fiabilité, car elles peuvent ne pas améliorer l'exactitude sous-jacente des réponses fournies par ces modèles.

A recent study introduced the concept of 'constraint tax' to measure the trade-offs between validity and correctness in structured outputs generated by small language models (SLMs), particularly focusing on models like Qwen2.5-0.5B and SmolLM2-1.7B. The research highlights that hard output constraints can lead to a false sense of reliability, as they may not improve the underlying accuracy of the answers provided by these models.

The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models

Was this article worth reading? Share it

Ready to build your own newsroom?