Reasoning via Video: The First Evaluation of Video Models' Reasoning Abilities through Maze-Solving Tasks

Kandinsky 5.0 has been introduced as a family of advanced foundation models designed for high-resolution image and video generation. This framework includes three main models: Kandinsky 5.0 Image Lite, a 6B parameter image generation model; Kandinsky 5.0 Video Lite, a lightweight 2B parameter text-to-video model; and Kandinsky 5.0 Video Pro, which features 19B parameters for superior video quality. The report also details the data curation lifecycle and innovative training techniques used in the model's development.

The paper presents a new metric called Physics-Constrained Multimodal Data Evaluation (PCMDE) aimed at improving the evaluation of multimodal synthetic images. Current metrics like BLEU and CIDEr often fail to accurately assess semantic and structural accuracy, particularly in specific domains. PCMDE integrates large language models with reasoning and vision-language models to enhance feature extraction, validation, and physics-guided reasoning.

HinTel-AlignBench is a newly proposed framework aimed at evaluating multilingual Vision-Language Models (VLMs) in Indian languages, specifically Hindi and Telugu, with English-aligned samples. The framework addresses limitations in current evaluations, such as reliance on unverified translations and narrow task coverage. It includes a semi-automated dataset creation process that combines back-translation and human verification, contributing to the advancement of equitable AI for low-resource languages.

The article discusses EyeVLA, a robotic eyeball designed for active visual perception in embodied AI systems. Unlike traditional models that passively process images, EyeVLA actively acquires detailed information while managing spatial constraints. This innovation aims to enhance the effectiveness of robotic applications in open-world environments by integrating action tokens with vision-language models (VLMs) for improved understanding and interaction.

DepthVision is a multimodal framework designed to enhance Vision-Language Models (VLMs) by utilizing LiDAR data without requiring architectural modifications or retraining. It synthesizes RGB-like images from sparse LiDAR point clouds using a conditional GAN and integrates a Luminance-Aware Modality Adaptation (LAMA) module to dynamically adjust image quality based on ambient lighting. This innovation aims to improve the reliability of autonomous vehicles in challenging visual conditions, such as darkness or motion blur.

Recent advancements in vision-language models (VLMs) have utilized large language models (LLMs) to achieve performance comparable to proprietary systems like GPT-4V. However, deploying these models on resource-constrained devices poses challenges due to high computational requirements. To address this, a new framework called Generation after Recalibration (GenRecal) has been introduced, which distills knowledge from large VLMs into smaller, more efficient models by aligning feature representations across diverse architectures.

Recent advancements in autonomous driving have investigated the application of vision-language models (VLMs) in visual question answering (VQA) frameworks for driving decision-making. However, these methods often rely on handcrafted prompts and exhibit inconsistent performance, which hampers their effectiveness in real-world scenarios. This study assesses state-of-the-art open-source VLMs on high-level decision-making tasks using ego-view visual inputs, revealing significant limitations in their ability to provide reliable, context-aware decisions.

The article presents a new framework called GMAT, which enhances Multiple Instance Learning (MIL) for whole slide image (WSI) classification. By integrating vision-language models (VLMs), GMAT aims to improve the generation of clinical descriptions that are more expressive and medically specific. This addresses limitations in existing methods that rely on large language models (LLMs) for generating descriptions, which often lack domain grounding and detailed medical specificity, thus improving alignment with visual features.