arXiv:2511.11005v3 Announce Type: replace 
Abstract: While recent Large Vision-Language Models (LVLMs) exhibit strong multimodal reasoning abilities, they often produce ungrounded or hallucinated responses because they rely too heavily on linguistic priors instead of visual evidence. This limitation highlights the absence of a quantitative measure of how much these models actually use visual information during reasoning. We propose Draft and Refine (DnR), an agent framework driven by a question-conditioned utilization metric. The metric quantifies the model's reliance on visual evidence by first constructing a query-conditioned relevance map to localize question-specific cues and then measuring dependence through relevance-guided probabilistic masking. Guided by this metric, the DnR agent refines its initial draft using targeted feedback from external visual experts. Each expert's output (such as boxes or masks) is rendered as visual cues on the image, and the model is re-queried to select the response that yields the largest improvement in utilization. This process strengthens visual grounding without retraining or architectural changes. Experiments across VQA and captioning benchmarks show consistent accuracy gains and reduced hallucination, demonstrating that measuring visual utilization provides a principled path toward more interpretable and evidence-driven multimodal agent systems. Code is available at https://github.com/EavnJeong/Draft-and-Refine-with-Visual-Experts.

تم اقتراح إطار عمل جديد يسمى Draft and Refine (DnR) لتحسين أداء نماذج اللغة والرؤية الكبيرة (LVLM) من خلال قياس اعتمادها على الأدلة البصرية أثناء التفكير. يستخدم هذا الإطار مقياس استخدام مشروط بالأسئلة لتوجيه النموذج في تحسين مخرجاته بناءً على التغذية الراجعة المستهدفة من الخبراء البصريين.

Se ha propuesto un nuevo marco llamado Draft and Refine (DnR) para mejorar el rendimiento de los Modelos de Lenguaje-Vision (LVLM) al cuantificar su dependencia de la evidencia visual durante el razonamiento. Este marco utiliza una métrica de utilización condicionada por preguntas para guiar al modelo en el refinamiento de sus salidas basándose en la retroalimentación específica de expertos visuales.

Un nouveau cadre appelé Draft and Refine (DnR) a été proposé pour améliorer la performance des grands modèles de vision-langage (LVLM) en quantifiant leur dépendance à l'évidence visuelle lors du raisonnement. Ce cadre utilise une métrique d'utilisation conditionnée par des questions pour guider le modèle dans le raffinement de ses résultats en fonction des retours ciblés d'experts visuels.

A new framework called Draft and Refine (DnR) has been proposed to enhance the performance of Large Vision-Language Models (LVLMs) by quantifying their reliance on visual evidence during reasoning. This framework utilizes a question-conditioned utilization metric to guide the model in refining its outputs based on targeted feedback from visual experts.