arXiv:2509.21882v3 Announce Type: replace 
Abstract: Reinforcement learning with verifiable rewards (RLVR) is a practical, scalable way to improve large language models on math, code, and other structured tasks. However, we argue that many headline RLVR gains are not yet well validated because reports often conflate policy improvement with three confounds: (i) budget mismatch between RLVR and baseline evaluations, (ii) attempt inflation and calibration drift that convert abstentions into confident answers, and (iii) benchmark data contamination. Using budget-matched reproductions and partial-prompt contamination probes, we find that several widely cited gaps shrink substantially or disappear once budgets, prompts, and dataset versions are matched and contaminated sets are treated as memorization probes rather than evidence of reasoning. This does not mean that RLVR is ineffective, but it implies that current measurements often overstate capability gains and obscure reliability costs. We therefore propose a compact, tax-aware minimum standard for RLVR training and evaluation: budget-matched saturation curves with variance, calibration, and abstention tracking, a judge-robustness stress test when LLM judges are used, and an explicit contamination screen. With these controls, RLVR remains effective and deployable in verifiable domains, but reasoning gains should be treated as provisional without them.

دراسة حديثة نُشرت على arXiv تناقش التكاليف الخفية وفجوات القياس المرتبطة بتعلم التعزيز مع المكافآت القابلة للتحقق (RLVR)، مشيرة إلى أن العديد من المكاسب المبلغ عنها قد تكون مبالغًا فيها بسبب مشكلات مثل عدم توافق الميزانية وتلوث البيانات. تشير النتائج إلى أنه عندما يتم محاذاة التقييمات بشكل صحيح، قد تتقلص المزايا المدركة لـ RLVR بشكل كبير.

Un estudio reciente publicado en arXiv analiza los costos ocultos y las brechas de medición asociadas con el aprendizaje por refuerzo con recompensas verificables (RLVR), destacando que muchas de las ganancias reportadas pueden estar exageradas debido a problemas como desajustes presupuestarios y contaminación de datos. Los hallazgos sugieren que, cuando las evaluaciones están correctamente alineadas, las ventajas percibidas del RLVR pueden disminuir significativamente.

Une étude récente publiée sur arXiv aborde les coûts cachés et les lacunes de mesure associés à l'apprentissage par renforcement avec des récompenses vérifiables (RLVR), soulignant que de nombreux gains rapportés pourraient être exagérés en raison de problèmes tels que des décalages budgétaires et une contamination des données. Les résultats suggèrent qu'une fois les évaluations correctement alignées, les avantages perçus du RLVR peuvent diminuer considérablement.

A recent study published on arXiv discusses the hidden costs and measurement gaps associated with reinforcement learning with verifiable rewards (RLVR), highlighting that many reported gains may be overstated due to issues like budget mismatches and data contamination. The findings suggest that when evaluations are properly aligned, the perceived advantages of RLVR may diminish significantly.