arXiv:2605.28008v1 Announce Type: cross 
Abstract: Large language models (LLMs) can now solve complex problems through long chain-of-thought (CoT) reasoning, but the trade-off between performance and token cost remains a central challenge. To address this issue, supervised fine-tuning (SFT) often uses compressed reasoning data, where CoT traces are shortened into compact forms. However, the effect of such compressed reasoning data on post-training remains poorly understood. In this paper, we propose a taxonomy of CoT consisting of Explicit CoT, which outputs all operations without aggregation, Composed CoT, which combines multiple operations into a single step, and Implicit CoT, which omits intermediate operations. We construct a synthetic compositional reasoning task that allows controlled variation of difficulty, compression granularity, and data size, and conducted a comprehensive set of experiments across different model families and sizes. Notably, we find that (i) coarser CoT requires more SFT data, (ii) compared with Explicit CoT, Composed CoT and Implicit CoT benefit more from data scaling, while Composed CoT benefits from data repetition and Implicit CoT tends to lead to memorization, (iii) unlike SFT, subsequent reinforcement learning (RL) with verifiable rewards (RLVR) decomposes compressed steps learned during SFT, and (iv) unidirectional CoT ordering shows stronger generalization on longer sequential tasks. Our findings provide implications for CoT design under data resource constraints and offer important insights into the mechanisms of SFT and RL in LLM post-training.

دراسة حديثة تستكشف فعالية بيانات التفكير المضغوطة في النماذج اللغوية الكبيرة (LLMs) بعد التدريب، مع تسليط الضوء على تصنيف لأساليب سلسلة التفكير (CoT): CoT الصريح، المركب، والضمني. تشير الأبحاث إلى أن CoT الأكثر خشونة يمكن أن تؤثر على الأداء وتكلفة الرموز، وهو ما لا يزال يمثل تحديًا كبيرًا في هذا المجال.

Un estudio reciente explora la efectividad de los datos de razonamiento comprimidos en los grandes modelos de lenguaje (LLMs) después del entrenamiento, destacando una taxonomía de enfoques de Cadena de Pensamiento (CoT): CoT Explícito, Compuesto e Implícito. La investigación indica que CoT más gruesos pueden impactar el rendimiento y el costo de tokens, lo que sigue siendo un desafío significativo en el campo.

Une étude récente examine l'efficacité des données de raisonnement compressées dans les grands modèles de langage (LLMs) après l'entraînement, mettant en lumière une taxonomie des approches de la chaîne de pensée (CoT) : CoT explicite, composé et implicite. La recherche indique que des CoT plus grossiers peuvent affecter la performance et le coût des tokens, ce qui reste un défi majeur dans le domaine.

A recent study explores the effectiveness of compressed reasoning data in large language models (LLMs) post-training, highlighting a taxonomy of Chain-of-Thought (CoT) approaches: Explicit, Composed, and Implicit CoT. The research indicates that coarser CoT can impact performance and token cost, which remains a significant challenge in the field.