Stable and Efficient Single-Rollout RL for Multimodal Reasoning

A recent study has explored the integration of visual and textual information in Multimodal Large Language Models (MLLMs), revealing that visual-text fusion occurs at specific layers within these models rather than uniformly across the network. The research highlights a late-stage

A recent study has introduced a framework aimed at mitigating hallucination issues in Multimodal Large Language Models (MLLMs) during Reinforcement Learning (RL) optimization. The research identifies key factors contributing to hallucinations, including over-reliance on visual reasoning and insufficient exploration diversity. The proposed framework incorporates modules for caption feedback, diversity-aware sampling, and conflict regularization to enhance model reliability.

A new benchmark called KidVis has been introduced to evaluate the visual perceptual capabilities of Multimodal Large Language Models (MLLMs), specifically assessing their performance against that of 6-7 year old children across six atomic visual capabilities. The results reveal a significant performance gap, with human children scoring an average of 95.32 compared to GPT-5's score of 67.33.

A new method called PRISM has been introduced to optimize the selection of training data for Multimodal Large Language Models (MLLMs), addressing the redundancy in rapidly growing datasets that increases computational costs. This self-pruning intrinsic selection method aims to enhance efficiency without the need for extensive training or proxy-based inference techniques.

A recent study has revealed that the group-relative advantage estimator used in Reinforcement Learning from Verifier Rewards (RLVR) is biased, systematically underestimating advantages for difficult prompts while overestimating them for easier ones. This imbalance can lead to ineffective exploration and exploitation strategies in training large language models.

The introduction of Process Relative Policy Optimization (PRPO) aims to enhance policy optimization for large language models (LLMs) by aligning process rewards with outcome rewards, addressing the limitations of existing critic-free methods like GRPO. PRPO provides a more nuanced approach by segmenting reasoning sequences and normalizing feedback, which improves the accuracy of models such as Qwen2.5-Math-1.5B on tasks like MATH500.

A recent study introduced MoHoBench, a benchmark designed to assess the honesty of Multimodal Large Language Models (MLLMs) when confronted with unanswerable visual questions. This research highlights the need for a systematic evaluation of MLLMs' response behaviors, as their trustworthiness in generating content remains underexplored.