GrndCtrl: Grounding World Models via Self-Supervised Reward Alignment

The recent introduction of IC-World, a novel framework for shared world modeling, allows for the parallel generation of multiple videos from a set of input images, enhancing the synthesis of dynamic visual environments. This framework leverages the in-context generation capabilities of large video models and incorporates reinforcement learning techniques to ensure consistency in geometry and motion across generated outputs.

The introduction of Soft Adaptive Policy Optimization (SAPO) addresses challenges in reinforcement learning (RL) for large language models (LLMs), particularly in achieving stable and effective policy optimization. SAPO replaces hard clipping with a smooth, temperature-controlled gate that adapts off-policy updates while retaining valuable learning signals, enhancing both sequence coherence and token adaptability.

The introduction of the Entropy Importance Sampling Policy Optimization (ESPO) framework aims to enhance the stability and efficiency of large language model (LLM) reinforcement learning by addressing the trade-off between optimization granularity and training stability. ESPO utilizes predictive entropy to decompose sequences into groups, allowing for more effective training sample utilization and improved credit assignment for reasoning steps.