arXiv:2412.15209v2 Announce Type: replace-cross 
Abstract: Despite significant advancements in Large Vision-Language Models (LVLMs)' capabilities, existing pixel-grounding models operate in single-image settings, limiting their ability to perform detailed, fine-grained comparisons across multiple images. Conversely, current multi-image understanding models lack pixel-level grounding. Our work addresses this gap by introducing the task of multi-image pixel-grounded reasoning alongside PRIMA, an LVLM that integrates pixel-level grounding with robust multi-image reasoning to produce contextually rich, pixel-grounded explanations. Central to PRIMA is SQuARE, a vision module that injects cross-image relational context into compact query-based visual tokens before fusing them with the language backbone. To support training and evaluation, we curate M4SEG, a new multi-image reasoning segmentation benchmark consisting of $\sim$744K question-answer pairs that require fine-grained visual understanding across multiple images. PRIMA outperforms state-of-the-art baselines with $7.83\%$ and $11.25\%$ improvements in Recall and S-IoU, respectively. Ablation studies further demonstrate the effectiveness of the proposed SQuARE module in capturing cross-image relationships.

يمثل تقديم PRIMA، نموذج اللغة-الرؤية متعدد الصور، تقدمًا كبيرًا في مجال الذكاء الاصطناعي من خلال دمج التثبيت على مستوى البكسل مع التفكير متعدد الصور، مما يمكّن من إجراء مقارنات بصرية مفصلة عبر صور متعددة. تعالج هذه الابتكارات قيود النماذج الحالية التي تركز إما على الصور الفردية أو تفتقر إلى التثبيت على مستوى البكسل.

La introducción de PRIMA, un modelo de lenguaje-visión multi-imagen, marca un avance significativo en el campo de la inteligencia artificial al integrar el anclaje a nivel de píxel con el razonamiento multi-imagen, lo que permite comparaciones visuales detalladas entre múltiples imágenes. Esta innovación aborda las limitaciones de los modelos existentes que se centran en imágenes individuales o carecen de anclaje a nivel de píxel.

L'introduction de PRIMA, un modèle de langage-vision multi-image, représente une avancée significative dans le domaine de l'intelligence artificielle en intégrant le ancrage pixel avec le raisonnement multi-image, permettant des comparaisons visuelles détaillées entre plusieurs images. Cette innovation répond aux limitations des modèles existants qui se concentrent soit sur des images uniques, soit manquent d'ancrage au niveau des pixels.

The introduction of PRIMA, a Multi-Image Vision-Language Model, marks a significant advancement in the field of artificial intelligence by integrating pixel-level grounding with multi-image reasoning, enabling detailed visual comparisons across multiple images. This innovation addresses the limitations of existing models that either focus on single images or lack pixel-level grounding.

PRIMA: Multi-Image Vision-Language Models for Reasoning Segmentation

Was this article worth reading? Share it

Supametas.AI

LucidQuery AI

Leonardo AI