arXiv:2605.18851v1 Announce Type: new 
Abstract: Recent advances in Reinforcement Learning (RL) have underscored its potential for incentivizing reasoning capabilities of Large Language Models (LLMs). However, existing step-level efforts suffer from costly annotations that limit domain coverage, while scalar scores further impose an information bottleneck, offering insufficient semantic bandwidth to improve intermediate decisions. Alternative language-critique approaches, which rely on frozen or external critics, provide richer textual feedback but lack the scalability needed for sustained policy improvement. In this work, we propose language-driven stepwise trajectory redirection, termed as STRIDE, a novel training framework that shifts process supervision from scalar rewards to learnable stepwise language feedback. Specifically, we co-train a generator and a generative verifier using only outcome-based rewards, eliminating external annotations, while delivering sustained policy improvement through jointly aligned verifier training. The verifier's stepwise language critiques explicitly localize and explain failures, enabling the generator to redirect reasoning trajectories at intermediate steps toward alternative decisions. The trajectory redirection design guarantees harmless policy improvement, even under noisy or suboptimal verifier feedback. Experiments on diverse reasoning benchmarks show that STRIDE significantly outperforms state-of-the-art baselines, as well as achieving breakthroughs on zero-pass-rate problems where scalar methods yield no learning signal in our ablation studies, demonstrating the effectiveness of learnable stepwise language feedback for enhancing LLM reasoning.

تم تقديم STRIDE مؤخرًا، وهو إطار تدريب جديد لنماذج اللغة الكبيرة (LLMs)، حيث يتم تحويل التركيز من المكافآت القياسية إلى التغذية الراجعة اللغوية القابلة للتعلم خطوة بخطوة، مما يعزز قدرات التفكير لهذه النماذج. تهدف هذه الطريقة إلى التغلب على قيود التوصيفات المكلفة والاختناقات المعلوماتية المرتبطة بأساليب التعلم المعزز التقليدية.

La reciente introducción de STRIDE, un nuevo marco de entrenamiento para Modelos de Lenguaje Grande (LLMs), desplaza el enfoque de recompensas escalares a retroalimentación lingüística paso a paso aprendible, mejorando así las capacidades de razonamiento de estos modelos. Este enfoque busca superar las limitaciones de las anotaciones costosas y los cuellos de botella informativos asociados con los métodos tradicionales de aprendizaje por refuerzo.

La récente introduction de STRIDE, un nouveau cadre d'entraînement pour les modèles de langage de grande taille (LLMs), déplace l'accent des récompenses scalaires vers des retours linguistiques étape par étape apprenables, améliorant ainsi les capacités de raisonnement de ces modèles. Cette approche vise à surmonter les limitations des annotations coûteuses et des goulets d'étranglement informationnels associés aux méthodes traditionnelles d'apprentissage par renforcement.

The recent introduction of STRIDE, a novel training framework for Large Language Models (LLMs), shifts the focus from scalar rewards to learnable stepwise language feedback, enhancing the reasoning capabilities of these models. This approach aims to overcome the limitations of costly annotations and information bottlenecks associated with traditional reinforcement learning methods.