arXiv:2606.08048v1 Announce Type: cross 
Abstract: Diffusion language models (DLMs) offer substantial speed advantages through parallel decoding, but the lack of token dependencies limits generation quality compared to autoregressive (AR) models. Recent progress attempts to bridge the gap via importance sampling, with DLM being the proposal and AR being the target. However, due to the huge gap between their distributions, the sampling requires a large number of particles and is thus expensive to compute. In this paper, we introduce PoE-Bridge, a novel decoding framework that drastically improves generation speed and accuracy by introducing an intermediate distribution to bridge the gap. The distribution is constructed as a Product-of-Experts (PoE) of the DLM proposal and the AR target. With the intermediate distribution, we first use the DLM to draft multiple continuations in parallel, then apply rejection sampling to verify the drafted tokens and move the resulting candidates toward the PoE. We then use importance sampling to further correct the PoE-aligned candidates toward the AR target. We further propose several improved techniques, including mixed-temperature sampling for enhanced diversity and elastic rejection windows for reducing wasted verification. Empirically, PoE-Bridge achieves significantly improved accuracy with $5\times$ speedup over the standard DLM decoding approach, and recovers at least 95% of the target AR model's performance, efficiently advancing most of the quality gap on challenging mathematical reasoning and coding tasks. Our code is available at https://github.com/juntongshi48/poe-bridge.

تم تقديم إطار جديد للتشفير يسمى PoE-Bridge لتحسين أداء نماذج اللغة الانتشارية (DLM) من خلال سد الفجوة مع النماذج الذاتية الانحدار (AR). يتيح هذا الإطار التشفير المتوازي مع تحسين سرعة ودقة التوليد من خلال توزيع وسيط يتشكل بواسطة نهج منتج الخبراء.

Se ha introducido un nuevo marco de decodificación llamado PoE-Bridge para mejorar el rendimiento de los modelos de lenguaje de difusión (DLM) al cerrar la brecha con los modelos autorregresivos (AR). Este marco permite la decodificación paralela mientras mejora la velocidad y precisión de generación a través de una distribución intermedia formada por un enfoque de Producto de Expertos.

Un nouveau cadre de décodage appelé PoE-Bridge a été introduit pour améliorer les performances des modèles de langage de diffusion (DLM) en comblant le fossé avec les modèles autorégressifs (AR). Ce cadre permet un décodage parallèle tout en améliorant la vitesse et la précision de génération grâce à une distribution intermédiaire formée par une approche de Produit d'Experts.

A novel decoding framework called PoE-Bridge has been introduced to enhance the performance of diffusion language models (DLMs) by bridging the gap with autoregressive (AR) models. This framework allows for parallel decoding while improving generation speed and accuracy through an intermediate distribution formed by a Product-of-Experts approach.