arXiv:2412.08619v3 Announce Type: replace 
Abstract: Physical reasoning remains a significant challenge for Vision-Language Models (VLMs). This limitation arises from an inability to translate learned knowledge into predictions about physical behavior. Although continual fine-tuning can mitigate this issue, it is expensive for large models and impractical to perform repeatedly for every task. This necessitates the creation of modular and scalable ways to teach VLMs about physical reasoning. To that end, we introduce Physics Context Builders (PCBs), a modular framework where specialized smaller VLMs are fine-tuned to generate detailed physical scene descriptions. These can be used as physical contexts to enhance the reasoning capabilities of larger VLMs. PCBs enable the separation of visual perception from reasoning, allowing us to analyze their relative contributions to physical understanding. We perform experiments on CLEVRER and on Falling Tower, a stability detection dataset with both simulated and real-world scenes, to demonstrate that PCBs provide substantial performance improvements, increasing average accuracy by up to 13.8% on complex physical reasoning tasks. Notably, PCBs also show strong Sim2Real transfer, successfully generalizing from simulated training data to real-world scenes.

إطار جديد يسمى Physics Context Builders يهدف إلى تحسين التفكير الفيزيائي في نماذج الرؤية-اللغة (VLM)، مما يعالج تحديًا رئيسيًا في هذا المجال. يمكن أن تكون الطرق التقليدية لضبط هذه النماذج مكلفة وغير عملية، خاصة للتطبيقات واسعة النطاق. تقدم هذه الطريقة المبتكرة حلاً معياريًا وقابلًا للتوسع، مما يسهل تعليم VLMs حول السلوك الفيزيائي. هذا التطور مهم لأنه قد يؤدي إلى نماذج أكثر دقة وكفاءة، مما يحسن في النهاية أدائها في التطبيقات الواقعية.

Un nuevo marco llamado Physics Context Builders tiene como objetivo mejorar el razonamiento físico en los Modelos de Visión-Lenguaje (VLM), abordando un desafío clave en el campo. Los métodos tradicionales de ajuste fino de estos modelos pueden ser costosos e imprácticos, especialmente para aplicaciones a gran escala. Este enfoque innovador ofrece una solución modular y escalable, facilitando la enseñanza a los VLM sobre el comportamiento físico. Este desarrollo es significativo, ya que podría llevar a modelos más precisos y eficientes, mejorando en última instancia su rendimiento en aplicaciones del mundo real.

Un nouveau cadre appelé Physics Context Builders vise à améliorer le raisonnement physique dans les modèles de vision-langage (VLM), abordant un défi clé dans le domaine. Les méthodes traditionnelles de réglage fin de ces modèles peuvent être coûteuses et peu pratiques, surtout pour des applications à grande échelle. Cette approche innovante offre une solution modulaire et évolutive, facilitant l'enseignement aux VLM sur le comportement physique. Ce développement est important car il pourrait conduire à des modèles plus précis et efficaces, améliorant finalement leurs performances dans des applications réelles.

A new framework called Physics Context Builders aims to enhance physical reasoning in Vision-Language Models (VLMs), addressing a key challenge in the field. Traditional methods of fine-tuning these models can be costly and impractical, especially for large-scale applications. This innovative approach offers a modular and scalable solution, making it easier to teach VLMs about physical behavior. This development is significant as it could lead to more accurate and efficient models, ultimately improving their performance in real-world applications.

Physics Context Builders: A Modular Framework for Physical Reasoning in Vision-Language Models

arXiv:2601.06204v2 Announce Type: replace 
Abstract: Intelligent anomaly detection in dynamic visual environments requires reconciling real-time performance with semantic interpretability. Conventional approaches address only fragments of this challenge. Reconstruction-based models capture low-level deviations without contextual reasoning, object detectors provide speed but limited semantics, and large vision-language systems deliver interpretability at prohibitive computational cost. This work introduces a cascading multi-agent framework that unifies these complementary paradigms into a coherent and interpretable architecture. Early modules perform reconstruction-gated filtering and object-level assessment, while higher-level reasoning agents are selectively invoked to interpret semantically ambiguous events. The system employs adaptive escalation thresholds and a publish-subscribe communication backbone, enabling asynchronous coordination and scalable deployment across heterogeneous hardware. Extensive evaluation on large-scale monitoring data demonstrates that the proposed cascade achieves a threefold reduction in latency compared to direct vision-language inference, while maintaining high perceptual fidelity (PSNR = 38.3 dB, SSIM = 0.965) and consistent semantic labeling. The framework advances beyond conventional detection pipelines by combining early-exit efficiency, adaptive multi-agent reasoning, and explainable anomaly attribution, establishing a reproducible and energy-efficient foundation for scalable intelligent visual monitoring.

تم تقديم إطار عمل جديد للكشف عن الشذوذ متعدد الوكلاء في أنظمة المراقبة، باستخدام نماذج الرؤية-اللغة وتصنيف قائم على التضمين لتحسين الأداء في الوقت الحقيقي وقابلية التفسير الدلالي. تتكامل هذه الطريقة مع منهجيات مختلفة، بما في ذلك التصفية المعتمدة على إعادة البناء والتقييم على مستوى الكائن، لمعالجة تعقيدات الكشف عن الشذوذ في البيئات البصرية الديناميكية.

Se ha introducido un nuevo marco para la detección de anomalías en cascada mediante múltiples agentes en sistemas de vigilancia, utilizando modelos de visión-lenguaje y clasificación basada en embeddings para mejorar el rendimiento en tiempo real y la interpretabilidad semántica. Este enfoque integra diversas metodologías, incluyendo filtrado basado en reconstrucción y evaluaciones a nivel de objeto, para abordar las complejidades de la detección de anomalías en entornos visuales dinámicos.

Un nouveau cadre pour la détection d'anomalies en cascade à plusieurs agents dans les systèmes de surveillance a été introduit, utilisant des modèles de vision-langage et une classification basée sur l'embedding pour améliorer la performance en temps réel et l'interprétabilité sémantique. Cette approche intègre diverses méthodologies, y compris le filtrage basé sur la reconstruction et les évaluations au niveau des objets, pour répondre aux complexités de la détection d'anomalies dans des environnements visuels dynamiques.

A new framework for cascading multi-agent anomaly detection in surveillance systems has been introduced, utilizing vision-language models and embedding-based classification to enhance real-time performance and semantic interpretability. This approach integrates various methodologies, including reconstruction-gated filtering and object-level assessments, to address the complexities of detecting anomalies in dynamic visual environments.

Cascading multi-agent anomaly detection in surveillance systems via vision-language models and embedding-based classification

arXiv:2508.13680v3 Announce Type: replace-cross 
Abstract: We introduce VMMU, a Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark designed to evaluate how vision-language models (VLMs) interpret and reason over visual and textual information beyond English. VMMU consists of 2.5k multimodal questions across 7 tasks, covering a diverse range of problem contexts, including STEM problem solving, data interpretation, rule-governed visual reasoning, and abstract visual reasoning. All questions require genuine multimodal integration, rather than reliance on text-only cues or OCR-based shortcuts. We evaluate a diverse set of state-of-the-art proprietary and open-source VLMs on VMMU. Despite strong Vietnamese OCR performance, proprietary models achieve only 66% mean accuracy. Further analysis shows that the primary source of failure is not OCR, but instead multimodal grounding and reasoning over text and visual evidence. Code and data are available at https://vmmu.github.io.

تم تقديم VMMU، وهو معيار فيتنامي لفهم وتقييم المهام المتعددة متعددة الوسائط، بهدف تقييم قدرات نماذج اللغة والرؤية (VLM) في تفسير واستنتاج المعلومات المرئية والنصية باللغة الفيتنامية. يتضمن هذا المعيار 2.5 ألف سؤال متعدد الوسائط عبر سبع مهام متنوعة، مع التركيز على التكامل الحقيقي متعدد الوسائط بدلاً من الاعتماد على الإشارات النصية فقط.

La introducción de VMMU, un referente vietnamita para la evaluación de la comprensión y el razonamiento multimodal, tiene como objetivo evaluar las capacidades de los modelos de lenguaje y visión (VLM) para interpretar y razonar sobre información visual y textual en vietnamita. Este referente incluye 2.5k preguntas multimodales en siete tareas diversas, enfatizando la integración multimodal genuina en lugar de depender únicamente de pistas textuales.

L'introduction de VMMU, une référence vietnamienne pour l'évaluation de la compréhension et du raisonnement multimodal, vise à évaluer les capacités des modèles de langage et de vision (VLM) à interpréter et raisonner sur des informations visuelles et textuelles en vietnamien. Cette référence comprend 2,5k questions multimodales réparties sur sept tâches diverses, mettant l'accent sur une véritable intégration multimodale plutôt que sur des indices uniquement textuels.

The introduction of VMMU, a Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark, aims to assess the capabilities of vision-language models (VLMs) in interpreting and reasoning over visual and textual information in Vietnamese. This benchmark includes 2.5k multimodal questions across seven diverse tasks, emphasizing genuine multimodal integration rather than text-only cues.

VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark

One More Thing in AI – Your Shortcut to AI Mastery

Physics Context Builders: A Modular Framework for Physical Reasoning in Vision-Language Models

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

MyFramework

The Visualizer

Guidejar-4eb95b

AIPortalX

Ready to build your own newsroom?