arXiv:2509.18711v2 Announce Type: replace 
Abstract: Remote sensing visual grounding (RSVG) aims to localize objects in remote sensing images based on free-form natural language expressions. Existing approaches are typically constrained to closed-set vocabularies, limiting their applicability in open-world scenarios. While recent attempts to leverage generic foundation models for open-vocabulary RSVG, they overly rely on expensive high-quality datasets and time-consuming fine-tuning. To address these limitations, we propose \textbf{RSVG-ZeroOV}, a training-free framework that aims to explore the potential of frozen generic foundation models for zero-shot open-vocabulary RSVG. Specifically, RSVG-ZeroOV comprises three key stages: (i) Overview: We utilize a vision-language model (VLM) to obtain cross-attention\footnote[1]{In this paper, although decoder-only VLMs use self-attention over all tokens, we refer to the image-text interaction part as cross-attention to distinguish it from pure visual self-attention.}maps that capture semantic correlations between text queries and visual regions. (ii) Focus: By leveraging the fine-grained modeling priors of a diffusion model (DM), we fill in gaps in structural and shape information of objects, which are often overlooked by VLM. (iii) Evolve: A simple yet effective attention evolution module is introduced to suppress irrelevant activations, yielding purified segmentation masks over the referred objects. Without cumbersome task-specific training, RSVG-ZeroOV offers an efficient and scalable solution. Extensive experiments demonstrate that the proposed framework consistently outperforms existing weakly-supervised and zero-shot methods.

يقدم إطار RSVG-ZeroOV نهجًا خاليًا من التدريب لتحديد المواقع البصرية ذات المفردات المفتوحة في الصور المستشعرة عن بُعد. تعالج هذه الطريقة القيود المفروضة على النماذج الحالية التي تعتمد على مفردات مغلقة ومجموعات بيانات مكلفة، مما يسمح بتطبيق أوسع في سيناريوهات العالم المفتوح. من خلال الاستفادة من نماذج الأساس المجمدة، يهدف RSVG-ZeroOV إلى تحسين تحديد مواقع الكائنات بناءً على تعبيرات اللغة الطبيعية دون الحاجة إلى ضبط دقيق مكثف.

El marco RSVG-ZeroOV introduce un enfoque sin entrenamiento para la anclaje visual de vocabulario abierto en cero disparos en imágenes de teledetección. Este método aborda las limitaciones de los modelos existentes que dependen de vocabularios cerrados y conjuntos de datos costosos, permitiendo una mayor aplicabilidad en escenarios del mundo abierto. Al aprovechar modelos de fundación genéricos congelados, RSVG-ZeroOV busca mejorar la localización de objetos basándose en expresiones en lenguaje natural sin necesidad de un ajuste fino extenso.

Le cadre RSVG-ZeroOV propose une approche sans entraînement pour le ancrage visuel à vocabulaire ouvert en zéro-shot dans les images de télédétection. Cette méthode répond aux limitations des modèles existants qui dépendent de vocabulaires fermés et de jeux de données coûteux, permettant une applicabilité plus large dans des scénarios ouverts. En s'appuyant sur des modèles de fondation génériques gelés, RSVG-ZeroOV vise à améliorer la localisation d'objets en fonction d'expressions en langage naturel sans nécessiter un ajustement fin étendu.

The RSVG-ZeroOV framework introduces a training-free approach for zero-shot open-vocabulary visual grounding in remote sensing images. This method addresses limitations of existing models that rely on closed-set vocabularies and expensive datasets, enabling broader applicability in open-world scenarios. By leveraging frozen generic foundation models, RSVG-ZeroOV aims to enhance the localization of objects based on natural language expressions without the need for extensive fine-tuning.

RSVG-ZeroOV: Exploring a Training-Free Framework for Zero-Shot Open-Vocabulary Visual Grounding in Remote Sensing Images

One More Thing in AI – Your Shortcut to AI Mastery

RSVG-ZeroOV: Exploring a Training-Free Framework for Zero-Shot Open-Vocabulary Visual Grounding in Remote Sensing Images

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Attentive AI

OpenL Translator

The Visualizer

Lenso.ai

Ready to build your own newsroom?