Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens

A new method for enhancing social interaction understanding in videos has been proposed, focusing on the alignment of verbal and non-verbal cues in multi-speaker scenarios. This approach addresses the limitations observed in existing Multimodal Large Language Models (MLLMs), which struggle with cross-modal attention consistency in such contexts.

VCU-Bridge has been introduced as a framework aimed at enhancing hierarchical visual connotation understanding in multimodal large language models (MLLMs). This framework addresses the limitations of current models that often process visual information in isolation, lacking the ability to integrate low-level perception with high-level reasoning. The accompanying HVCU-Bench benchmark is designed to evaluate this new approach effectively.

A novel vulnerability in vision-language models (VLMs) has been identified through the introduction of IAG, a method that enables multi-target backdoor attacks on VLM-based visual grounding systems. This technique utilizes dynamically generated, input-aware triggers that are text-guided, allowing for imperceptible manipulation of visual inputs while maintaining normal performance on benign samples.

Researchers have introduced L2V-CoT, a novel training-free approach that facilitates the transfer of Chain-of-Thought (CoT) reasoning from large language models (LLMs) to Vision-Language Models (VLMs) using Linear Artificial Tomography (LAT). This method addresses the challenges VLMs face in multi-step reasoning tasks due to limited multimodal reasoning data.

A new dataset named VisReason has been introduced to enhance visual Chain-of-Thought (CoT) reasoning in multimodal large language models (MLLMs). Comprising 489,000 annotated examples across four domains, VisReason aims to facilitate complex reasoning by providing multi-round, human-like rationales that guide MLLMs through visual reasoning steps. Additionally, a subset called VisReason-Pro, featuring 165,000 examples, has been curated with expert-level annotations.

A recent study has proposed a new framework called Subspace Projection Debiasing (SPD) to address the pervasive demographic biases in Vision-Language Models (VLMs). This framework challenges the traditional post-hoc debiasing methods that focus on coordinate-wise adjustments, revealing that biases are distributed across linear subspaces rather than isolated coordinates.

A new framework called ReVeL (Rewrite and Verify by LLM) has been proposed to enhance the multiple-choice question answering (MCQA) format used in evaluating multimodal language models. This framework transforms MCQA into open-form questions while ensuring answers remain verifiable, addressing issues of answer guessing and unreliable accuracy metrics during reinforcement fine-tuning (RFT).

The introduction of Perceptual-Evidence Anchored Reinforced Learning (PEARL) marks a significant advancement in multimodal reasoning, addressing the limitations of traditional Reinforcement Learning with Verifiable Rewards (RLVR) in Vision-Language Models (VLMs). PEARL enhances reasoning by anchoring it to verified visual evidence, thus mitigating issues like visual hallucinations and reward hacking.