arXiv:2506.06220v2 Announce Type: replace 
Abstract: Vision-language models (VLMs) have shown strong performance on text-to-image retrieval benchmarks. However, bridging this success to real-world applications remains a challenge. In practice, human search behavior is rarely a one-shot action. Instead, it is often a multi-round process guided by clues in mind. That is, a mental image ranging from vague recollections to vivid mental representations of the target image. Motivated by this gap, we study the task of Mental Image Retrieval (MIR), which targets the realistic yet underexplored setting where users refine their search for a mentally envisioned image through multi-round interactions with an image search engine. Central to successful interactive retrieval is the capability of machines to provide users with clear, actionable feedback; however, existing methods rely on indirect or abstract verbal feedback, which can be ambiguous, misleading, or ineffective for users to refine the query. To overcome this, we propose GenIR, a generative multi-round retrieval paradigm leveraging diffusion-based image generation to explicitly reify the AI system's understanding at each round. These synthetic visual representations provide clear, interpretable feedback, enabling users to refine their queries intuitively and effectively. We further introduce a fully automated pipeline to generate a high-quality multi-round MIR dataset. Experimental results demonstrate that GenIR significantly outperforms existing interactive methods in the MIR scenario. This work establishes a new task with a dataset and an effective generative retrieval method, providing a foundation for future research in this direction

يعد التطور الأخير لنظام GenIR، الذي يوفر تغذية بصرية توليدية لاسترجاع الصور الذهنية، تقدمًا كبيرًا في مجال نماذج الرؤية واللغة. على عكس عمليات البحث عن الصور التقليدية التي تتم في خطوة واحدة، يدرك GenIR أن سلوك البحث البشري غالبًا ما يكون تكراريًا ويتأثر بالصور الذهنية. يمكن أن تعزز هذه الابتكار كيفية تفاعلنا مع التكنولوجيا، مما يجعل استرجاع الصور أكثر حدسية وفعالية. مع استمرارنا في سد الفجوة بين قدرات الذكاء الاصطناعي والتطبيقات الواقعية، يمكن أن يحول GenIR العديد من القطاعات، من التعليم إلى الصناعات الإبداعية، من خلال تحسين كيفية العثور على المعلومات المرئية واستخدامها.

El reciente desarrollo de GenIR, un sistema de retroalimentación visual generativa para la recuperación de imágenes mentales, marca un avance significativo en el campo de los modelos de visión-lenguaje. A diferencia de las búsquedas de imágenes tradicionales de una sola vez, GenIR reconoce que el comportamiento de búsqueda humano es a menudo iterativo e influenciado por la imaginación mental. Esta innovación podría mejorar nuestra interacción con la tecnología, haciendo que la recuperación de imágenes sea más intuitiva y efectiva. A medida que continuamos cerrando la brecha entre las capacidades de la IA y las aplicaciones del mundo real, GenIR podría transformar varios sectores, desde la educación hasta las industrias creativas, al mejorar nuestra forma de encontrar y utilizar información visual.

Le développement récent de GenIR, un système de retour visuel génératif pour la récupération d'images mentales, marque une avancée significative dans le domaine des modèles de vision-langage. Contrairement aux recherches d'images traditionnelles en une seule étape, GenIR reconnaît que le comportement de recherche humain est souvent itératif et influencé par l'imagerie mentale. Cette innovation pourrait améliorer notre interaction avec la technologie, rendant la récupération d'images plus intuitive et efficace. Alors que nous continuons à combler le fossé entre les capacités de l'IA et les applications réelles, GenIR pourrait transformer divers secteurs, de l'éducation aux industries créatives, en améliorant notre façon de trouver et d'utiliser des informations visuelles.

The recent development of GenIR, a generative visual feedback system for mental image retrieval, marks a significant advancement in the field of vision-language models. Unlike traditional one-shot image searches, GenIR recognizes that human search behavior is often iterative and influenced by mental imagery. This innovation could enhance how we interact with technology, making image retrieval more intuitive and effective. As we continue to bridge the gap between AI capabilities and real-world applications, GenIR could transform various sectors, from education to creative industries, by improving how we find and utilize visual information.

GenIR: Generative Visual Feedback for Mental Image Retrieval

arXiv:2512.15372v1 Announce Type: cross 
Abstract: Vision transformers in vision-language models apply uniform computational effort across all images, expending 175.33 GFLOPs (ViT-L/14) whether analysing a straightforward product photograph or a complex street scene. We propose ICAR (Image Complexity-Aware Retrieval), which enables vision transformers to use less compute for simple images whilst processing complex images through their full network depth. The key challenge is maintaining cross-modal alignment: embeddings from different processing depths must remain compatible for text matching. ICAR solves this through dual-path training that produces compatible embeddings from both reduced-compute and full-compute processing. This maintains compatibility between image representations and text embeddings in the same semantic space, whether an image exits early or processes fully. Unlike existing two-stage approaches that require expensive reranking, ICAR enables direct image-text matching without additional overhead. To determine how much compute to use, we develop ConvNeXt-IC, which treats image complexity assessment as a classification task. By applying modern classifier backbones rather than specialised architectures, ConvNeXt-IC achieves state-of-the-art performance with 0.959 correlation with human judgement (Pearson) and 4.4x speedup. Evaluated on standard benchmarks augmented with real-world web data, ICAR achieves 20% practical speedup while maintaining category-level performance and 95% of instance-level performance, enabling sustainable scaling of vision-language systems.

تم اقتراح نهج جديد يسمى الاسترجاع التكيفي الواعي بتعقيد الصورة (ICAR) لتحسين نماذج الرؤية-اللغة من خلال السماح لمحوّلات الرؤية بتخصيص الموارد الحاسوبية بناءً على تعقيد الصورة. تتيح هذه الطريقة معالجة الصور البسيطة بموارد أقل مع ضمان تحليل الصور المعقدة بالتفصيل، مما يحافظ على التوافق بين الأنماط النصية والصورية لتحقيق مطابقة فعالة.

Se ha propuesto un nuevo enfoque llamado Recuperación Adaptativa Consciente de la Complejidad de la Imagen (ICAR) para mejorar los modelos de visión-lenguaje, permitiendo que los transformadores de visión asignen recursos computacionales según la complejidad de la imagen. Este método permite procesar imágenes simples con menos recursos mientras asegura que las imágenes complejas se analicen en su totalidad, manteniendo la alineación intermodal para un emparejamiento de texto efectivo.

Une nouvelle approche appelée Image Complexity-Aware Retrieval (ICAR) a été proposée pour améliorer les modèles de vision-langage en permettant aux transformateurs de vision d'allouer des ressources informatiques en fonction de la complexité de l'image. Cette méthode permet de traiter des images simples avec moins de ressources tout en garantissant que les images complexes sont analysées en détail, maintenant ainsi l'alignement intermodal pour un appariement texte efficace.

A new approach called Image Complexity-Aware Retrieval (ICAR) has been proposed to enhance vision-language models by allowing vision transformers to allocate computational resources based on image complexity. This method enables simpler images to be processed with less compute while ensuring that complex images are analyzed in full detail, maintaining cross-modal alignment for effective text matching.

Image Complexity-Aware Adaptive Retrieval for Efficient Vision-Language Models

One More Thing in AI – Your Shortcut to AI Mastery

GenIR: Generative Visual Feedback for Mental Image Retrieval

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

4o Image Gen

Imagerr AI

MemeGen AI

Lenso.ai

Ready to build your own newsroom?