ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection

Recent research has focused on improving the honesty of language models in deductive reasoning, emphasizing their ability to provide answers only when logically entailed by the premises. The study introduces multi-step tasks and datasets to evaluate this capability, revealing that existing training methods struggle with these challenges.

A new framework called ReVeL (Rewrite and Verify by LLM) has been proposed to enhance the multiple-choice question answering (MCQA) format used in evaluating multimodal language models. This framework transforms MCQA into open-form questions while ensuring answers remain verifiable, addressing issues of answer guessing and unreliable accuracy metrics during reinforcement fine-tuning (RFT).

A recent study highlights the limitations of Direct Preference Optimization (DPO) in diffusion models, particularly the issue of likelihood displacement, where the probabilities of preferred samples decrease during training. This phenomenon can lead to suboptimal performance in video generation tasks, which are increasingly relevant in AI applications.

A new study introduces Periodic Asynchrony as a method to enhance on-policy reinforcement learning, addressing the inefficiencies of synchronous execution in mainstream frameworks. By separating inference and training, this approach allows for independent scaling of components while maintaining accuracy equivalent to traditional methods.

The introduction of VADE, a Variance-Aware Dynamic Sampling framework, aims to enhance group-based policy optimization methods in multimodal reinforcement learning (RL) by addressing the gradient vanishing problem. This issue arises when identical rewards are assigned to all responses within a group, leading to diminished training signals. VADE proposes an online sample-level difficulty estimation to improve the selection of effective samples during training.

The introduction of Perceptual-Evidence Anchored Reinforced Learning (PEARL) marks a significant advancement in multimodal reasoning, addressing the limitations of traditional Reinforcement Learning with Verifiable Rewards (RLVR) in Vision-Language Models (VLMs). PEARL enhances reasoning by anchoring it to verified visual evidence, thus mitigating issues like visual hallucinations and reward hacking.

The introduction of Bayesian Prior-Guided Optimization (BPGO) enhances Group Relative Policy Optimization (GRPO) by addressing the inherent ambiguity in visual generation tasks. BPGO incorporates a semantic prior anchor to model reward uncertainty, allowing for more effective optimization by emphasizing reliable feedback while down-weighting ambiguous signals.

A new method called Syn-GRPO (Synthesis-GRPO) has been proposed to enhance the reinforcement learning capabilities of Multimodal Large Language Models (MLLMs) by synthesizing high-quality training data through an online data generator. This approach aims to address the existing challenges of low data quality that limit the exploration scope in MLLM training.