arXiv:2510.21007v2 Announce Type: replace 
Abstract: Chain-of-thought (CoT) prompting has emerged as a common technique for enhancing the reasoning abilities of large language models (LLMs). While extended reasoning can boost accuracy on complex tasks, it is often unnecessary and substantially increases token usage, limiting the practicality of reasoning models in many scenarios. Recent models, such as GPT-OSS and Qwen3, expose controls that enable users to adjust the length of CoT or determine whether it is used at all. Yet, it remains unclear when CoT should be used: on some tasks it improves performance, while on others it provides little benefit or even harms performance. We address this challenge with confidence-gated CoT, where a model invokes reasoning only when confidence in its direct answer is low. To this end, we present the first systematic study of training-free confidence estimation methods for CoT gating. Specifically, we evaluate four training-free confidence estimation methods and compare them to a random baseline and an oracle that always knows when CoT is needed. Through extensive experiments, we show that existing training-free confidence measures can reduce redundant CoT and outperform randomly invoked CoT. However, the utility of individual confidence measures is inconsistent, varying with both the dataset and the model, underscoring the difficulty of deploying confidence-gated CoT in practice. By analysing both strengths and failure modes, our study highlights the potential and limitations of current methods and paves the way toward more reliable adaptive gating of CoT.

تتناول دراسة حديثة استخدام تقنية سلسلة التفكير (CoT) في نماذج اللغة الكبيرة (LLMs) مثل GPT-OSS و Qwen3. بينما يمكن أن تعزز تقنية CoT التفكير والدقة في المهام المعقدة، فإنها غالبًا ما تؤدي إلى استخدام غير ضروري للتوكنات، مما قد يعيق التطبيقات العملية. تبرز الدراسة أهمية تقديرات الثقة في تحديد متى يكون استخدام CoT ضروريًا حقًا، بهدف تحسين التوازن بين عمق التفكير والكفاءة. هذا الأمر مهم لأنه قد يحسن من قابلية استخدام نماذج اللغة الكبيرة في سيناريوهات متعددة، مما يجعلها أدوات أكثر فعالية للمستخدمين.

Un estudio reciente discute el uso de la técnica de cadena de pensamiento (CoT) en modelos de lenguaje grandes (LLMs) como GPT-OSS y Qwen3. Aunque el CoT puede mejorar el razonamiento y la precisión en tareas complejas, a menudo conduce a un uso innecesario de tokens, lo que puede obstaculizar las aplicaciones prácticas. La investigación destaca la importancia de las estimaciones de confianza para decidir cuándo es realmente necesario el CoT, buscando optimizar el equilibrio entre la profundidad del razonamiento y la eficiencia. Esto es significativo, ya que podría mejorar la usabilidad de los LLMs en varios escenarios, convirtiéndolos en herramientas más efectivas para los usuarios.

Une étude récente aborde l'utilisation de l'invite de chaîne de pensée (CoT) dans les grands modèles de langage (LLMs) tels que GPT-OSS et Qwen3. Bien que le CoT puisse améliorer le raisonnement et la précision pour des tâches complexes, il entraîne souvent une utilisation excessive de jetons, ce qui peut entraver les applications pratiques. La recherche souligne l'importance des estimations de confiance pour décider quand le CoT est vraiment nécessaire, visant à optimiser l'équilibre entre la profondeur du raisonnement et l'efficacité. Cela est significatif car cela pourrait améliorer l'utilisabilité des LLMs dans divers scénarios, les rendant plus efficaces pour les utilisateurs.

A recent study discusses the use of chain-of-thought (CoT) prompting in large language models (LLMs) like GPT-OSS and Qwen3. While CoT can enhance reasoning and accuracy for complex tasks, it often leads to unnecessary token usage, which can hinder practical applications. The research highlights the importance of confidence estimates in deciding when CoT is truly needed, aiming to optimize the balance between reasoning depth and efficiency. This is significant as it could improve the usability of LLMs in various scenarios, making them more effective tools for users.

Can Confidence Estimates Decide When Chain-of-Thought Is Necessary for LLMs?

arXiv:2508.16151v2 Announce Type: replace-cross 
Abstract: The rapid advancement of Large Language Models (LLMs) has established language as a core general-purpose cognitive substrate, driving the demand for specialized Language Processing Units (LPUs) tailored for LLM inference. To overcome the growing energy consumption of LLM inference systems, this paper proposes a Hardwired-Neurons Language Processing Unit (HNLPU), which physically hardwires LLM weight parameters into the computational fabric, achieving several orders of magnitude computational efficiency improvement by extreme specialization. However, a significant challenge still lies in the scale of modern LLMs. A straightforward hardwiring of gpt-oss 120 B would require fabricating photomask sets valued at over 6 billion dollars, rendering this straightforward solution economically impractical. Addressing this challenge, we propose the novel Metal-Embedding methodology. Instead of embedding weights in a 2D grid of silicon device cells, Metal-Embedding embeds weight parameters into the 3D topology of metal wires. This brings two benefits: (1) a 15x increase in density, and (2) 60 out of 70 photomask layers are homogeneous across chips, including all EUV photomasks. In total, Metal-Embedding reduced the photomask cost by 112x, bringing the Non-Recurring Engineering (NRE) cost of HNLPU into an economically viable range. Experimental results show that HNLPU achieved 249,960 tokens/s (5,555x/85x that of GPU/WSE), 36 tokens/J (1,047x/283x that of GPU/WSE), 13,232 mm2 total die area, $59.46 M-123.5 M estimated NRE at 5 nm technology. Analysis shows that HNLPU achieved 41.7-80.4x improvement in cost-effectiveness and 357x reduction in carbon footprint compared to OpenAI-scale H100 clusters, under an annual weight updating assumption.

يهدف تطوير وحدات معالجة اللغة ذات الخلايا العصبية الموصلة (HNLPU) إلى تعزيز كفاءة نماذج اللغة الكبيرة (LLMs) من خلال توصيل معلمات الوزن فعليًا في الهيكل الحسابي، مما يحقق تحسينًا كبيرًا في الكفاءة الحسابية. ومع ذلك، تواجه الجدوى الاقتصادية لهذا النهج تحديات بسبب التكاليف العالية المرتبطة بتصنيع مجموعات القناع الضوئي لنماذج اللغة الكبيرة الحديثة، مثل gpt-oss 120 B.

El desarrollo de las Unidades de Procesamiento de Lenguaje con Neuronas Cableadas (HNLPU) tiene como objetivo mejorar la eficiencia de los Modelos de Lenguaje Grande (LLM) al cablear físicamente los parámetros de peso en la estructura computacional, logrando una mejora significativa en la eficiencia computacional. Sin embargo, la viabilidad económica de este enfoque se ve desafiada por los altos costos asociados con la fabricación de juegos de fotomáscaras para los LLM modernos, como gpt-oss 120 B.

Le développement des unités de traitement du langage à neurones câblés (HNLPU) vise à améliorer l'efficacité des modèles de langage de grande taille (LLM) en câblant physiquement les paramètres de poids dans le tissu computationnel, ce qui améliore considérablement l'efficacité computationnelle. Cependant, la faisabilité économique de cette approche est remise en question par les coûts élevés associés à la fabrication de jeux de photomasques pour les LLM modernes, tels que gpt-oss 120 B.

The development of Hardwired-Neurons Language Processing Units (HNLPU) aims to enhance the efficiency of Large Language Models (LLMs) by physically hardwiring weight parameters into the computational fabric, significantly improving computational efficiency. However, the economic feasibility of this approach is challenged by the high costs associated with fabricating photomask sets for modern LLMs, such as gpt-oss 120 B.

Can Confidence Estimates Decide When Chain-of-Thought Is Necessary for LLMs?

Was this article worth reading? Share it

LucidQuery AI

PromptKit

Sellm

GPT Rules

LCW

ChatOne

Ready to build your own newsroom?