arXiv:2506.00789v3 Announce Type: replace 
Abstract: Retrieval-Augmented Generation (RAG) enhances recency and factuality in answers. However, existing evaluations rarely test how well these systems cope with real-world noise, conflicting between internal and external retrieved contexts, or fast-changing facts. We introduce Retrieval-Aware Robustness Evaluation (RARE), a unified framework and large-scale benchmark that jointly stress-tests query and document perturbations over dynamic, time-sensitive corpora. One of the central features of RARE is a knowledge-graph-driven synthesis pipeline (RARE-Get) that automatically extracts single and multi-hop relations from the customized corpus and generates multi-level question sets without manual intervention. Leveraging this pipeline, we construct a dataset (RARE-Set) spanning 527 expert-level time-sensitive finance, economics, and policy documents and 48295 questions whose distribution evolves as the underlying sources change. To quantify resilience, we formalize retrieval-conditioned robustness metrics (RARE-Met) that capture a model's ability to remain correct or recover when queries, documents, or real-world retrieval results are systematically altered. Our findings reveal that RAG systems are unexpectedly sensitive to perturbations. Moreover, they consistently demonstrate lower robustness on multi-hop queries compared to single-hop queries across all domains.

تم تقديم إطار جديد يسمى تقييم القوة الواعي للاسترجاع (RARE) لتحسين تقييم أنظمة الجيل المعزز بالاسترجاع (RAG). يتناول هذا الإطار الحاجة الملحة لاختبار كيفية تعامل هذه الأنظمة مع التحديات الواقعية، مثل الضوضاء والمعلومات المتضاربة. من خلال توفير معيار واسع النطاق يركز على البيانات الديناميكية والحساسة للوقت، يهدف RARE إلى تحسين موثوقية ودقة الاستجابات التي تولدها الذكاء الاصطناعي، مما يمثل تقدمًا كبيرًا في مجال الذكاء الاصطناعي واسترجاع المعلومات.

Se ha introducido un nuevo marco llamado Evaluación de Robustez Consciente de la Recuperación (RARE) para mejorar la evaluación de los sistemas de Generación Aumentada por Recuperación (RAG). Este marco aborda la necesidad crítica de probar cómo estos sistemas manejan los desafíos del mundo real, como el ruido y la información contradictoria. Al proporcionar un estándar a gran escala que se centra en datos dinámicos y sensibles al tiempo, RARE busca mejorar la fiabilidad y precisión de las respuestas generadas por IA, lo que representa un avance significativo en el campo de la IA y la recuperación de información.

Un nouveau cadre appelé Évaluation de Robustesse Consciente de la Récupération (RARE) a été introduit pour améliorer l'évaluation des systèmes de Génération Augmentée par Récupération (RAG). Ce cadre répond au besoin crucial de tester comment ces systèmes gèrent les défis du monde réel, tels que le bruit et les informations contradictoires. En fournissant une référence à grande échelle axée sur des données dynamiques et sensibles au temps, RARE vise à améliorer la fiabilité et la précision des réponses générées par l'IA, ce qui constitue une avancée significative dans le domaine de l'IA et de la récupération d'informations.

A new framework called Retrieval-Aware Robustness Evaluation (RARE) has been introduced to enhance the evaluation of Retrieval-Augmented Generation (RAG) systems. This framework addresses the critical need for testing how these systems handle real-world challenges, such as noise and conflicting information. By providing a large-scale benchmark that focuses on dynamic and time-sensitive data, RARE aims to improve the reliability and accuracy of AI-generated responses, making it a significant advancement in the field of AI and information retrieval.

RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems

arXiv:2601.08747v1 Announce Type: new 
Abstract: Current context augmentation methods, such as retrieval-augmented generation, are essential for solving knowledge-intensive reasoning tasks.However, they typically adhere to a rigid, brute-force strategy that executes retrieval at every step. This indiscriminate approach not only incurs unnecessary computational costs but also degrades performance by saturating the context with irrelevant noise. To address these limitations, we introduce Agentic Context Evolution (ACE), a framework inspired by human metacognition that dynamically determines whether to seek new evidence or reason with existing knowledge. ACE employs a central orchestrator agent to make decisions strategically via majority voting.It aims to alternate between activating a retriever agent for external retrieval and a reasoner agent for internal analysis and refinement. By eliminating redundant retrieval steps, ACE maintains a concise and evolved context. Extensive experiments on challenging multi-hop QA benchmarks demonstrate that ACE significantly outperforms competitive baselines in accuracy while achieving efficient token consumption.Our work provides valuable insights into advancing context-evolved generation for complex, knowledge-intensive tasks.

تشير التطورات الأخيرة في أساليب تعزيز السياق، وخاصة تقديم تطور السياق الوكيني (ACE)، إلى إطار ديناميكي يوازن بين استرجاع الأدلة والتفكير، مما يعزز المهام المعقدة التي تتطلب المعرفة. يهدف ACE إلى تحسين الأداء من خلال اتخاذ قرارات استراتيجية حول متى يجب البحث عن معلومات جديدة أو الاعتماد على المعرفة الموجودة، مما يقلل من التكاليف الحاسوبية والضوضاء في السياق.

Los recientes avances en métodos de aumento de contexto, especialmente la introducción de la Evolución de Contexto Agentivo (ACE), proponen un marco dinámico que equilibra la recuperación de evidencia y el razonamiento, mejorando así las tareas de razonamiento intensivo en conocimiento. ACE busca optimizar el rendimiento al decidir estratégicamente cuándo buscar nueva información o confiar en el conocimiento existente, reduciendo así los costos computacionales y el ruido en el contexto.

Des avancées récentes dans les méthodes d'augmentation de contexte, notamment l'introduction de l'Évolution de Contexte Agentique (ACE), proposent un cadre dynamique qui équilibre la récupération d'évidence et le raisonnement, améliorant ainsi les tâches de raisonnement intensif en connaissances. ACE vise à optimiser la performance en décidant stratégiquement quand récupérer de nouvelles informations ou s'appuyer sur des connaissances existantes, réduisant ainsi les coûts computationnels et le bruit dans le contexte.

Recent advancements in context augmentation methods, particularly the introduction of Agentic Context Evolution (ACE), propose a dynamic framework that balances evidence retrieval and reasoning, enhancing knowledge-intensive reasoning tasks. ACE aims to optimize performance by strategically deciding when to retrieve new information or rely on existing knowledge, thereby reducing computational costs and noise in the context.

To Retrieve or To Think? An Agentic Approach for Context Evolution

One More Thing in AI – Your Shortcut to AI Mastery

RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Airparser

ZeroGPT.org

GPTHumanizer

Research AI

Ready to build your own newsroom?