arXiv:2506.06295v3 Announce Type: replace 
Abstract: Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models. Recently, a new paradigm has emerged in the form of diffusion-based Large Language Models (dLLMs), which generate text by iteratively denoising masked segments. This approach has shown significant advantages and potential. However, dLLMs suffer from high inference latency. Traditional ARM acceleration techniques, such as Key-Value caching, are incompatible with dLLMs due to their bidirectional attention mechanism. To address this specific challenge, our work begins with a key observation that dLLM inference involves a static prompt and a partially dynamic response, where most tokens remain stable across adjacent denoising steps. Based on this, we propose dLLM-Cache, a training-free adaptive caching framework that combines long-interval prompt caching with partial response updates guided by feature similarity. This design enables efficient reuse of intermediate computations without compromising model performance. Extensive experiments on representative dLLMs, including LLaDA 8B and Dream 7B, show that dLLM-Cache achieves up to 9.1x FLOPs reduction on LongBench-HotpotQA while maintaining competitive output quality. Notably, our method brings dLLM inference latency close to that of ARMs under many settings. The code for this work is publicly available at: https://github.com/maomaocun/dLLM-cache.

يمثل تقديم dLLM-Cache تقدمًا كبيرًا في كفاءة نماذج اللغة الكبيرة القائمة على الانتشار (dLLMs)، حيث يتناول تحدي زمن الاستدلال العالي من خلال استخدام إطار تخزين مؤقت تكيفي يجمع بين تخزين الموجهات على فترات طويلة وتحديثات جزئية للاستجابة.

La introducción de dLLM-Cache marca un avance significativo en la eficiencia de los Modelos de Lenguaje de Difusión (dLLMs), abordando el desafío de la alta latencia de inferencia mediante un marco de caché adaptativa que combina la caché de prompts de largo intervalo con actualizaciones parciales de respuestas.

L'introduction de dLLM-Cache représente une avancée significative dans l'efficacité des modèles de langage à diffusion (dLLMs), en s'attaquant au défi de la latence d'inférence élevée grâce à un cadre de mise en cache adaptatif qui combine la mise en cache de prompts à long intervalle avec des mises à jour de réponses partielles.

The introduction of dLLM-Cache marks a significant advancement in the efficiency of diffusion-based Large Language Models (dLLMs), addressing the challenge of high inference latency by utilizing an adaptive caching framework that combines long-interval prompt caching with partial response updates.