arXiv:2605.29886v1 Announce Type: new 
Abstract: Retrieval-augmented generation (RAG) improves knowledge-intensive question answering by incorporating external evidence. However, existing RAG methods still suffer from hallucinations and subtle reasoning errors. Recent studies introduce external critics to refine RAG outputs, yet they often provide coarse-grained and weakly structured feedback, exhibit over-aggressive intervention, and lead to noisy and unreliable refinement, limiting their effectiveness for correction. To tackle these issues, we propose CRITIC-R1, a structured critic framework that formulates and learns RAG critique as an explicit error diagnosis problem using reinforcement learning (RL). Our framework categorizes common RAG errors into multiple diagnostic dimensions, including verdict, error location, reasoning analysis, and fix generation. To learn these capabilities, we design two reward functions: Conservative Judgement Alignment (CJA) first encourages calibrated high-level judgements while mitigating the over-aggressive phenomenon, whereas Diagnostic Quality Alignment (DQA) further improves fine-grained diagnostic feedback through gated rewards. We train the critic model using GRPO-based RL with process-level supervision collected from external LLM teacher models. Experiments across five QA benchmarks show that CRITIC-R1 consistently improves answer quality over strong RAG baselines.
  Our source code is available at https://anonymous.4open.science/r/critic-r1-FCB0

تقدم CRITIC-R1 إطارًا نقديًا منظمًا يهدف إلى تحسين الجيل المعزز بالاسترجاع (RAG) من خلال معالجة الأخطاء الشائعة باستخدام التعلم المعزز. يصنف هذا الإطار أخطاء RAG إلى أبعاد تشخيصية، مما يسمح بتشخيص أكثر دقة وتغذية راجعة.

La introducción de CRITIC-R1 presenta un marco de críticos estructurados destinado a mejorar la Generación Aumentada por Recuperación (RAG) al abordar errores comunes mediante el aprendizaje por refuerzo. Este marco categoriza los errores de RAG en dimensiones diagnósticas, lo que permite un diagnóstico y retroalimentación más precisos.

L'introduction de CRITIC-R1 présente un cadre de critique structuré visant à améliorer la génération augmentée par récupération (RAG) en abordant les erreurs courantes grâce à l'apprentissage par renforcement. Ce cadre catégorise les erreurs de RAG en dimensions diagnostiques, permettant un diagnostic et un retour d'information plus précis.

The introduction of CRITIC-R1 presents a structured critic framework aimed at enhancing Retrieval-Augmented Generation (RAG) by addressing common errors through reinforcement learning. This framework categorizes RAG errors into diagnostic dimensions, allowing for more precise error diagnosis and feedback.