VADE: Variance-Aware Dynamic Sampling via Online Sample-Level Difficulty Estimation for Multimodal RL

Recent research has focused on improving the honesty of language models in deductive reasoning, emphasizing their ability to provide answers only when logically entailed by the premises. The study introduces multi-step tasks and datasets to evaluate this capability, revealing that existing training methods struggle with these challenges.

ExPO-HM (Explain-then-Detect Policy Optimization for Hateful Memes) has been proposed to enhance the detection of hateful memes, addressing limitations in existing models that primarily provide binary predictions without context. This new approach aims to incorporate reasoning similar to human annotators, improving the understanding of policy-relevant cues such as targets and attack types.

A new framework called ReVeL (Rewrite and Verify by LLM) has been proposed to enhance the multiple-choice question answering (MCQA) format used in evaluating multimodal language models. This framework transforms MCQA into open-form questions while ensuring answers remain verifiable, addressing issues of answer guessing and unreliable accuracy metrics during reinforcement fine-tuning (RFT).

A new study introduces Periodic Asynchrony as a method to enhance on-policy reinforcement learning, addressing the inefficiencies of synchronous execution in mainstream frameworks. By separating inference and training, this approach allows for independent scaling of components while maintaining accuracy equivalent to traditional methods.

The SPINE framework introduces a token-selective approach to test-time reinforcement learning, addressing the challenges faced by large language models (LLMs) and multimodal LLMs (MLLMs) during distribution shifts at test-time. By focusing on high-entropy tokens and applying an entropy-band regularizer, SPINE aims to enhance model performance and maintain exploration during reinforcement learning processes.

The introduction of Perceptual-Evidence Anchored Reinforced Learning (PEARL) marks a significant advancement in multimodal reasoning, addressing the limitations of traditional Reinforcement Learning with Verifiable Rewards (RLVR) in Vision-Language Models (VLMs). PEARL enhances reasoning by anchoring it to verified visual evidence, thus mitigating issues like visual hallucinations and reward hacking.

The introduction of Bayesian Prior-Guided Optimization (BPGO) enhances Group Relative Policy Optimization (GRPO) by addressing the inherent ambiguity in visual generation tasks. BPGO incorporates a semantic prior anchor to model reward uncertainty, allowing for more effective optimization by emphasizing reliable feedback while down-weighting ambiguous signals.

A new method called Syn-GRPO (Synthesis-GRPO) has been proposed to enhance the reinforcement learning capabilities of Multimodal Large Language Models (MLLMs) by synthesizing high-quality training data through an online data generator. This approach aims to address the existing challenges of low data quality that limit the exploration scope in MLLM training.