Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning

GlobalRAG is a proposed reinforcement learning framework aimed at enhancing global reasoning in multi-hop question answering (QA). It addresses limitations in current methods by decomposing questions into subgoals, coordinating retrieval with reasoning, and refining evidence iteratively. The framework introduces new rewards to encourage coherent planning and reliable execution of subgoals, aiming to improve the effectiveness of multi-hop QA systems.

Distribution Matching Distillation (DMD) is a method that distills a pre-trained multi-step diffusion model into a few-step model to enhance inference efficiency. The proposed DMDR framework integrates Reinforcement Learning (RL) techniques into the distillation process, demonstrating that DMD loss serves as a more effective regularization method. This approach allows for simultaneous distillation and RL, improving the few-step generator's performance and visual quality.

Self-Examining Reinforcement Learning (SERL) is a proposed framework that addresses challenges in applying Reinforcement Learning (RL) to open-domain tasks. Traditional methods face issues with subjectivity and reliance on external rewards. SERL innovatively positions large language models (LLMs) as both Actor and Judge, utilizing internal reward mechanisms. It employs Copeland-style pairwise comparisons to enhance the Actor's capabilities and introduces a self-consistency reward to improve the Judge's reliability, aiming to advance RL applications in open domains.

The paper titled 'Agent-R1: Training Powerful LLM Agents with End-to-End Reinforcement Learning' discusses the potential of Large Language Models (LLMs) in creating agents that can interact with their environment to solve complex problems. It highlights the challenges in applying Reinforcement Learning (RL) to LLMs and the lack of tailored frameworks for training these agents. The authors propose a systematic extension of the Markov Decision Process (MDP) framework to define key components of LLM agents and introduce Agent-R1, a flexible training framework.

Seer is a new online context learning system designed to enhance the efficiency of synchronous reinforcement learning (RL) for large language models (LLMs). It addresses performance bottlenecks in existing RL systems, particularly during the rollout phase, which is hampered by long-tail latency and poor resource utilization. Seer employs techniques such as divided rollout, context-aware scheduling, and adaptive grouped speculative decoding to significantly improve throughput and resource efficiency, achieving a 74% to 97% increase in end-to-end rollout performance.