arXiv:2511.16160v1 Announce Type: new 
Abstract: Spatial intelligence is a critical frontier for Multimodal Large Language Models (MLLMs), empowering them to comprehend the physical world. Drawing inspiration from human perception mechanisms, existing studies attempt to construct a coherent spatial understanding via grid-based cognitive maps from multi-frame visual inputs. However, current grid-based map methods rely on discretized raster representations, which limit the model's ability in fine-grained spatial reasoning. To overcome this limitation, we propose Video2Layout, a framework for reconstructing metric-grounded spatial layouts from video. The framework employs continuous object boundary coordinates to quantify inter-object physical distances and object size. This empowers the model with quantitative spatial computation capabilities, effectively alleviating the inherent ambiguity when describing spatial relationships in natural language. Specifically, our method comprises two core stages. First, in supervised fine-tuning stage, we construct a high-quality dataset from the AI2THOR simulator, which enables the model to learn the mapping from visual inputs to precise boundary coordinates. Subsequently, a reinforcement fine-tuning stage further enhances the model's real-world generalization capabilities. To systematically evaluate the correlation between cognitive map accuracy and image quantity, as well as how the quantity of image inputs affects spatial reasoning accuracy, we introduce QVS-Bench, a diagnostic benchmark designed to analyze the relevant mechanisms. Evaluated on QVS-Bench and mainstream spatial reasoning benchmarks, our model, V2LO-7B achieves an average improvement of 4.92% over the model trained on grid maps, validating the superiority of our method. Our code is available at https://github.com/ybrrraway/Video2Layout.

Video2Layout هو إطار مقترح يهدف إلى تعزيز التفكير المكاني في نماذج اللغة متعددة الوسائط (MLLMs) من خلال إعادة بناء خرائط معرفية قائمة على القياس من مدخلات الفيديو. تستخدم هذه الطريقة إحداثيات حدود الكائنات المستمرة لقياس المسافات الفيزيائية وأحجام الكائنات، مما يعالج القيود المفروضة على الطرق المعتمدة على الشبكات التي تعيق التفكير المكاني الدقيق. يهدف الإطار إلى تحسين قدرة النموذج على وصف العلاقات المكانية بلغة طبيعية.

Video2Layout es un marco propuesto que busca mejorar el razonamiento espacial en Modelos de Lenguaje Multimodal (MLLM) al reconstruir mapas cognitivos anclados en métricas a partir de entradas de video. Este enfoque utiliza coordenadas de contorno de objetos continuas para medir distancias físicas y tamaños de objetos, abordando las limitaciones de los métodos basados en cuadrículas que obstaculizan el razonamiento espacial detallado. El marco tiene como objetivo mejorar la capacidad del modelo para describir relaciones espaciales en lenguaje natural.

Video2Layout est un cadre proposé visant à améliorer le raisonnement spatial dans les modèles de langage multimodaux (MLLM) en reconstruisant des cartes cognitives ancrées dans des métriques à partir d'entrées vidéo. Cette approche utilise des coordonnées de contours d'objets continues pour mesurer les distances physiques et les tailles d'objets, répondant ainsi aux limitations des méthodes basées sur des grilles qui entravent le raisonnement spatial détaillé. Le cadre vise à améliorer la capacité du modèle à décrire les relations spatiales en langage naturel.

Video2Layout is a proposed framework aimed at enhancing spatial reasoning in Multimodal Large Language Models (MLLMs) by reconstructing metric-grounded cognitive maps from video inputs. This approach utilizes continuous object boundary coordinates to measure physical distances and object sizes, addressing limitations of current grid-based methods that hinder fine-grained spatial reasoning. The framework aims to improve the model's ability to describe spatial relationships in natural language.

Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoning

arXiv:2511.12861v3 Announce Type: replace-cross 
Abstract: With the remarkable success of Multimodal Large Language Models (MLLMs) in perception tasks, enhancing their complex reasoning capabilities has emerged as a critical research focus. Existing models still suffer from challenges such as opaque reasoning paths and insufficient generalization ability. Chain-of-Thought (CoT) reasoning, which has demonstrated significant efficacy in language models by enhancing reasoning transparency and output interpretability, holds promise for improving model reasoning capabilities when extended to the multimodal domain. This paper provides a systematic review centered on "Multimodal Chain-of-Thought" (MCoT). First, it analyzes the background and theoretical motivations for its inception from the perspectives of technical evolution and task demands. Then, it introduces mainstream MCoT methods from three aspects: CoT paradigms, the post-training stage, and the inference stage, while also analyzing their underlying mechanisms. Furthermore, the paper summarizes existing evaluation benchmarks and metrics, and discusses the application scenarios of MCoT. Finally, it analyzes the challenges currently facing MCoT and provides an outlook on its future research directions.

أظهرت التطورات الأخيرة في نماذج اللغة متعددة الوسائط (MLLM) الحاجة إلى تعزيز قدراتها على التفكير، وخاصة من خلال نموذج Chain-of-Thought (CoT). تهدف هذه المقاربة إلى تحسين شفافية التفكير وقابلية التفسير، مع معالجة التحديات الحالية مثل مسارات التفكير غير الواضحة والقدرات المحدودة على التعميم. توفر المراجعة المنهجية لطرق Multimodal Chain-of-Thought (MCoT) رؤى حول أسسها النظرية وتطبيقاتها العملية.

Los recientes avances en Modelos de Lenguaje Multimodal (MLLM) han resaltado la necesidad de mejorar sus capacidades de razonamiento, particularmente a través del paradigma Chain-of-Thought (CoT). Este enfoque busca mejorar la transparencia del razonamiento y la interpretabilidad, abordando desafíos existentes como los caminos de razonamiento opacos y las limitaciones en la capacidad de generalización. La revisión sistemática de los métodos de Multimodal Chain-of-Thought (MCoT) proporciona información sobre sus fundamentos teóricos y aplicaciones prácticas.

Les avancées récentes dans les modèles de langage multimodaux (MLLM) ont mis en évidence la nécessité d'améliorer leurs capacités de raisonnement, notamment par le biais du paradigme Chain-of-Thought (CoT). Cette approche vise à améliorer la transparence et l'interprétabilité du raisonnement, en s'attaquant aux défis existants tels que les chemins de raisonnement opaques et les capacités de généralisation limitées. La revue systématique des méthodes de Multimodal Chain-of-Thought (MCoT) fournit des informations sur leurs fondements théoriques et leurs applications pratiques.

Recent advancements in Multimodal Large Language Models (MLLMs) have highlighted the need to enhance their reasoning capabilities, particularly through the Chain-of-Thought (CoT) paradigm. This approach aims to improve reasoning transparency and interpretability, addressing existing challenges such as opaque reasoning paths and limited generalization abilities. The systematic review of Multimodal Chain-of-Thought (MCoT) methods provides insights into their theoretical foundations and practical applications.

Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoning

Was this article worth reading? Share it

Dyad

Com.locatelloapp

TypeThinkAI