arXiv:2512.02044v1 Announce Type: new 
Abstract: Diffusion Language Models (DLMs) have recently achieved significant success due to their any-order generation capabilities. However, existing inference methods typically rely on local, immediate-step metrics such as confidence or entropy which inherently lack a more reliable perspective. This limitation frequently leads to inconsistent sampling trajectories and suboptimal generation quality. To address this, we propose Coherent Contextual Decoding (CCD), a novel inference framework built upon two core innovations. First, CCD employs a trajectory rectification mechanism that leverages historical context to enhance sequence coherence, enabling the early rejection of suboptimal paths. We demonstrate that this mechanism is theoretically equivalent to modeling the consistency of historical steps via the conditional mutual information between context and token predictions. Building on this theoretical insight, we further address the inefficiency of conventional uniform decoding budgets. Instead of rigid allocations based on diffusion steps, we introduce an adaptive sampling strategy that dynamically adjusts the unmasking budget for each step according to our consistency metric. Consequently, our method significantly improves the quality of generation trajectories while accelerating the sampling process. Empirically, our method achieves a simultaneous enhancement in both inference speed and performance across diverse benchmarks on Dream and LLaDA, delivering up to 3.48x speedup alongside 3.91% performance improvement.

أدت التطورات الأخيرة في نماذج اللغة الانتشارية (DLM) إلى تقديم مفهوم فك الشفرات السياقية المتماسكة (CCD)، وهو إطار استدلالي جديد مصمم لتحسين تماسك التسلسلات من خلال استخدام السياق التاريخي. يهدف هذا النهج إلى تصحيح مسارات العينة وتحسين جودة التوليد من خلال رفض المسارات غير المثلى في المراحل المبكرة من العملية.

Los avances recientes en los Modelos de Lenguaje de Difusión (DLM) han llevado a la introducción de la Decodificación Contextual Coherente (CCD), un nuevo marco de inferencia diseñado para mejorar la coherencia de las secuencias utilizando el contexto histórico. Este enfoque busca rectificar las trayectorias de muestreo y mejorar la calidad de generación al rechazar caminos subóptimos en las primeras etapas del proceso.

Des avancées récentes dans les modèles de langage de diffusion (DLM) ont conduit à l'introduction du décodage contextuel cohérent (CCD), un nouveau cadre d'inférence conçu pour améliorer la cohérence des séquences en utilisant le contexte historique. Cette approche vise à rectifier les trajectoires d'échantillonnage et à améliorer la qualité de génération en rejetant tôt les chemins non optimaux.

Recent advancements in Diffusion Language Models (DLMs) have led to the introduction of Coherent Contextual Decoding (CCD), a new inference framework designed to enhance sequence coherence by utilizing historical context. This approach aims to rectify sampling trajectories and improve generation quality by rejecting suboptimal paths early in the process.

Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models

Was this article worth reading? Share it

LucidQuery AI

Langfuse

LCW

Dubsmart LLC

Polidict

Https

Ready to build your own newsroom?