Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline

AdaTok introduces an innovative object-level token merging strategy for Adaptive Token compression, aimed at enhancing the efficiency of Multimodal Large Language Models (MLLMs). Traditional patch-level tokenization has resulted in excessive computational and memory demands, leading to misalignments with human cognitive processes. The proposed method significantly reduces token usage to 10% while maintaining nearly 96% of the original model's performance, addressing critical challenges in multimodal understanding and reasoning.

Recent advancements in multimodal large language models (MLLMs) have significantly improved vision-language understanding. However, their high computational demands hinder their use in resource-limited environments like robotics and personal assistants. Traditional Transformer-based methods face efficiency challenges due to quadratic complexity, and smaller models often fail to capture critical visual details for fine-grained reasoning tasks. Viper-F1 introduces a Hybrid State-Space Vision-Language Model that utilizes Liquid State-Space Dynamics and a Token-Grid Correlation Module to enhance e…

VIR-Bench is a new benchmark designed to evaluate the geospatial and temporal understanding of multimodal large language models (MLLMs) through the reconstruction of travel video itineraries. It consists of 200 travel videos, addressing a gap in current benchmarks that primarily focus on indoor or short-range outdoor activities. The study highlights the challenges faced by state-of-the-art MLLMs in handling extended geospatial-temporal trajectories, which are crucial for real-world applications like AI planning and navigation.

DenseAnnotate is an innovative online annotation platform designed to facilitate the creation of dense, fine-grained annotations for images and 3D scenes through spoken descriptions. As multimodal large language models (MLLMs) gain traction, the demand for high-quality, task-centered training data has surged. Current datasets often rely on sparse annotations, which inadequately capture the visual content of images. DenseAnnotate addresses this gap by allowing annotators to narrate their observations, enhancing the expressiveness and speed of the annotation process.

The rapid growth of e-commerce necessitates advanced multimodal models capable of understanding complex visual and textual product information. The proposed MOON2.0 framework addresses challenges faced by existing multimodal large language models (MLLMs) in representation learning, including modality imbalance, underutilization of intrinsic relationships between visual and textual data, and noise handling in e-commerce data. MOON2.0 features a Modality-driven Mixture-of-Experts module and a Dual-level Alignment method to enhance product understanding.

Multimodal Large Language Models (MLLMs) have demonstrated significant cross-modal capabilities but continue to struggle with hallucinations. To address this issue, VBackChecker has been introduced as a reference-free hallucination detection framework. This framework verifies the consistency of MLLM-generated responses with visual inputs using a pixel-level Grounding LLM that incorporates reasoning and segmentation capabilities. Additionally, a new pipeline for generating instruction-tuning data, R-Instruct, has been developed, enhancing interpretability and handling rich-context scenarios eff…