OptPO: Optimal Rollout Allocation for Test-time Policy Optimization

Kardia-R1 has introduced KardiaBench, a benchmark designed to enhance emotional reasoning in conversational agents by utilizing a dataset of 178,080 QA pairs from 671 real-world profiles, addressing the limitations of existing systems that lack personalized emotional understanding.

A new adaptive curriculum mechanism called CAPO (Curriculum Advantage Policy Optimization) has been proposed to enhance cross-domain reasoning tasks in reinforcement learning. This mechanism aims to improve reasoning capabilities by utilizing advantage signals, initially focusing on positive samples to establish a solid foundation before incorporating negative signals for better discrimination.

The introduction of SeeNav-Agent marks a significant advancement in Vision-Language Navigation (VLN) by addressing common errors in perception, reasoning, and planning that hinder navigation performance. This framework incorporates a dual-view Visual Prompt technique to enhance spatial understanding and a novel step-level Reinforcement Fine-Tuning method, Step Reward Group Policy Optimization (SRGPO), to improve navigation task rewards.

A new framework for risk-aware constrained reinforcement learning (RL) has been proposed, utilizing optimized certainty equivalents (OCEs) to address the shortcomings of traditional methods that overlook risky events in reward distributions. This approach ensures robustness in both reward values and time, providing a more comprehensive solution for high-stakes applications.

The introduction of Group Adaptive Policy Optimization (GAPO) addresses the challenges of skewed reward distributions in reinforcement learning for large language models (LLMs) used in code editing. GAPO employs an adaptive approach to compute advantage estimates by utilizing an outlier-free highest-density interval, enhancing the robustness of advantage calculations in real-world scenarios.

Recent advancements in video world modeling have led to the introduction of GrndCtrl, a self-supervised framework that aligns pretrained world models with geometric and perceptual rewards. This development aims to enhance the realism and utility of generative models in navigation tasks by ensuring spatial coherence and long-horizon stability.

The introduction of Soft Adaptive Policy Optimization (SAPO) addresses challenges in reinforcement learning (RL) for large language models (LLMs), particularly in achieving stable and effective policy optimization. SAPO replaces hard clipping with a smooth, temperature-controlled gate that adapts off-policy updates while retaining valuable learning signals, enhancing both sequence coherence and token adaptability.

The introduction of the Entropy Importance Sampling Policy Optimization (ESPO) framework aims to enhance the stability and efficiency of large language model (LLM) reinforcement learning by addressing the trade-off between optimization granularity and training stability. ESPO utilizes predictive entropy to decompose sequences into groups, allowing for more effective training sample utilization and improved credit assignment for reasoning steps.