DVPO: Distributional Value Modeling-based Policy Optimization for LLM Post-Training

A novel approach has been proposed to enhance the reasoning capabilities of large language models (LLMs) through offline goal-conditioned reinforcement learning (RL), addressing the limitations of current multi-turn RL training methods that are costly and inefficient. This method utilizes goal-conditioned value functions to predict task outcomes based on actions, enabling better planning and reasoning in complex tasks such as negotiation and persuasion.

OneThinker has been introduced as an all-in-one reasoning model that integrates image and video understanding across various visual tasks, including question answering and segmentation. This model aims to overcome the limitations of existing approaches that treat image and video reasoning as separate domains, thereby enhancing scalability and knowledge sharing.

A recent study investigates the impact of explicit world-modeling objectives on the internal representations and performance of Transformers, particularly in the context of a controlled Rubik's Cube task. The research compares standard next-token prediction with two world-modeling strategies, revealing that explicit modeling enhances representation quality and downstream performance after reinforcement learning post-training.

A recent study has introduced a novel method called ACraft for automatic attack discovery in Few-Shot Class-Incremental Learning (FSCIL) using Large Language Models (LLMs). This research highlights the challenges posed by traditional attack methods like PGD and FGSM, which either fail to effectively target base classes or require extensive expert knowledge, thus necessitating a specialized approach for FSCIL.

A comprehensive study has been conducted on the application of deep reinforcement learning (RL) algorithms for dynamic algorithm configuration (DAC), specifically focusing on optimizing the population size parameter of the (1+($\lambda$,$\lambda$))-GA on OneMax instances. The research identifies significant challenges such as scalability degradation and learning instability, attributed to under-exploration and planning horizon coverage.

AdaptVision has been introduced as a new paradigm in Vision-Language Models (VLMs), focusing on adaptive visual token acquisition to enhance efficiency in visual question answering tasks. By employing a coarse-to-fine approach, the model selectively acquires visual information as needed, addressing the computational overhead associated with traditional methods that rely on fixed-ratio compression.

The introduction of Group-relative Trajectory-based Policy Optimization (GTPO) aims to enhance the stability and performance of Group Relative Policy Optimization (GRPO) in training Large Language Models (LLMs). GTPO addresses critical issues such as conflicting gradient updates on valuable tokens and policy collapse, which have hindered effective model alignment and training processes. By amplifying positive feedback and filtering out high-entropy completions, GTPO seeks to improve convergence and reliability.

Kardia-R1 has introduced KardiaBench, a benchmark designed to enhance emotional reasoning in conversational agents by utilizing a dataset of 178,080 QA pairs from 671 real-world profiles, addressing the limitations of existing systems that lack personalized emotional understanding.