arXiv:2605.07210v2 Announce Type: replace-cross 
Abstract: This paper shows how diffusion language models (DLMs) can be used as effective and efficient retrievers. Existing DLM-based retrievers (e.g., DiffEmbed) follow BERT-style encoding, representing each query or passage as a single mean-pooled vector. This ignores how DLMs are trained to generate responses through masked-position prediction under bidirectional attention, a capability that can provide stronger retrieval signals. We propose DiffRetriever, which uses the DLM's native masked-position prediction directly for retrieval. For each query or passage, DiffRetriever appends one or more masked positions, using the outputs as retrieval representations in a single forward pass. With one masked position, single-representation DiffRetriever already improves over DiffEmbed on the same backbones. DiffRetriever also naturally extends to multi-representation retrieval: DLMs process multiple masked positions jointly, enabling ColBERT-style fine-grained matching with little additional encoding latency. In autoregressive LLM retrievers, the same multi-representation strategy requires sequential decoding and therefore incurs much higher latency. DiffRetriever obtains the strongest aggregate effectiveness within our matched comparison, outperforming DiffEmbed, PromptReps, and RepLLaMA. Masked-position counts selected on training data transfer well across datasets, while per-query variation suggests headroom for adaptive allocation. Code is available at https://github.com/ielab/diffretriever.

يمثل تقديم DiffRetriever تقدمًا كبيرًا في استخدام نماذج اللغة الانتشارية (DLM) لمهام الاسترجاع، مستفيدًا من قدراتها في التنبؤ بالمواقع المخفية لتحسين كفاءة الاسترجاع. على عكس النماذج السابقة التي استخدمت متجهات متوسطة، يستخدم DiffRetriever مواقع مخفية متعددة لتوليد تمثيلات استرجاع أكثر قوة في تمريرة واحدة للأمام.

La introducción de DiffRetriever marca un avance significativo en el uso de modelos de lenguaje de difusión (DLM) para tareas de recuperación, aprovechando sus capacidades de predicción de posiciones enmascaradas para mejorar la eficiencia de la recuperación. A diferencia de los modelos anteriores que utilizaban vectores promediados, DiffRetriever emplea múltiples posiciones enmascaradas para generar representaciones de recuperación más robustas en un solo pase hacia adelante.

L'introduction de DiffRetriever représente une avancée significative dans l'utilisation des modèles de langage de diffusion (DLM) pour les tâches de récupération, en tirant parti de leurs capacités de prédiction de positions masquées pour améliorer l'efficacité de la récupération. Contrairement aux modèles précédents qui utilisaient des vecteurs moyennés, DiffRetriever emploie plusieurs positions masquées pour générer des représentations de récupération plus robustes en un seul passage avant.

The introduction of DiffRetriever marks a significant advancement in the use of diffusion language models (DLMs) for retrieval tasks, leveraging their masked-position prediction capabilities to enhance retrieval efficiency. Unlike previous models that utilized mean-pooled vectors, DiffRetriever employs multiple masked positions to generate more robust retrieval representations in a single forward pass.