arXiv:2506.11083v3 Announce Type: replace 
Abstract: We introduce RedDebate, a novel multi-agent debate framework that provides the foundation for Large Language Models (LLMs) to identify and mitigate their unsafe behaviours. AI safety approaches often rely on costly human evaluation or isolated single-model assessment, both constrained by scalability and prone to oversight failures. RedDebate employs collaborative argumentation among multiple LLMs across diverse debate scenarios, enabling them to critically evaluate one another's reasoning and systematically uncover unsafe failure modes through fully automated red-teaming. To support this, we propose designing distinct long-term memory modules that preserve safety-relevant insights from debate interactions and leverage them during subsequent inference, facilitating continuous refinement of model behaviour. Empirical evaluation on safety benchmarks across a diverse set of models demonstrates that RedDebate substantially reduces unsafe outputs. While debate alone allows LLMs to refine their behaviour, the addition of memory yields further error reductions. To the best of our knowledge, RedDebate is the first fully automated framework to unify multi-agent debate and red-teaming to progressively enhance LLM safety without human intervention.

تم تقديم RedDebate كإطار عمل للنقاش متعدد الوكلاء مصمم لتحسين أمان نماذج اللغة الكبيرة (LLMs) من خلال تمكينها من تقييم تفكير بعضها البعض واكتشاف السلوكيات غير الآمنة من خلال اختبارات آلية. تهدف هذه المقاربة إلى معالجة قيود أساليب أمان الذكاء الاصطناعي التقليدية التي تعتمد على التقييم البشري أو تقييمات نموذج واحد، والتي يمكن أن تكون مكلفة وعرضة لفشل الإشراف.

RedDebate se ha presentado como un marco de debate multiagente diseñado para mejorar la seguridad de los Modelos de Lenguaje Grande (LLMs) al permitirles evaluar el razonamiento de otros y detectar comportamientos inseguros a través de pruebas automatizadas. Este enfoque busca abordar las limitaciones de los métodos tradicionales de seguridad en IA que dependen de la evaluación humana o de evaluaciones de un solo modelo, que pueden ser costosas y propensas a fallos de supervisión.

RedDebate a été introduit comme un cadre de débat multi-agents conçu pour améliorer la sécurité des modèles de langage de grande taille (LLMs) en leur permettant d'évaluer le raisonnement des autres et d'identifier les comportements dangereux par le biais de tests automatisés. Cette approche vise à remédier aux limites des méthodes de sécurité de l'IA traditionnelles qui reposent sur l'évaluation humaine ou des évaluations de modèles uniques, souvent coûteuses et sujettes à des erreurs de surveillance.

RedDebate has been introduced as a multi-agent debate framework designed to enhance the safety of Large Language Models (LLMs) by enabling them to evaluate each other's reasoning and identify unsafe behaviors through automated red-teaming. This approach aims to address the limitations of traditional AI safety methods that rely on human evaluation or single-model assessments, which can be costly and prone to oversight failures.