arXiv:2605.27186v1 Announce Type: new 
Abstract: Large language models often solve tasks from a fully specified prompt but degrade when the same requirements unfold over multiple turns, known as the lost-in-conversation (LiC) gap. We trace part of this degradation to self-contamination: intermediate assistant replies enter later context and carry early deviations forward. Motivated by this mechanism, we propose MAIGO, an on-policy self-distillation method that reduces this contamination using history-cleaned references from the model's own policy. For middle turns, MAIGO removes prior assistant replies while preserving the user-visible sharded prefix; for answer turns, it distills from paired full-view references conditioned on the completed user-side dialogue. A reliability weight downweights middle-turn samples that disagree with the clean reference. MAIGO requires no verifier rewards, state labels, or inference-time scaffolding. Under the LiC paired-view protocol with deterministic verifiers, MAIGO improves Qwen2.5-7B-Instruct SHARDED accuracy from 52.8 to 66.1 and the SHARDED/FULL ratio from 66.5% to 84.1%, while keeping FULL accuracy within 2.3 points. These results show that self-contamination is a trainable component of the LiC gap.

قدم الباحثون MAIGO، وهي طريقة مبتكرة للاستخلاص الذاتي على السياسة تهدف إلى التخفيف من فجوة الضياع في المحادثة (LiC) في نماذج اللغة الكبيرة (LLMs). تتناول هذه الطريقة مشكلة التلوث الذاتي، حيث تؤثر الردود السابقة للمساعد سلبًا على التفاعلات اللاحقة، من خلال استخدام مراجع نظيفة من التاريخ من سياسة النموذج نفسه.

Los investigadores han presentado MAIGO, un método innovador de auto-destilación en política diseñado para mitigar la brecha de pérdida en la conversación (LiC) en grandes modelos de lenguaje (LLMs). Este enfoque aborda el problema de la auto-contaminación, donde las respuestas anteriores del asistente influyen negativamente en las interacciones posteriores, utilizando referencias limpiadas de la historia de la propia política del modelo.

Des chercheurs ont introduit MAIGO, une méthode innovante d'auto-distillation sur politique visant à atténuer l'écart perdu dans la conversation (LiC) dans les grands modèles de langage (LLMs). Cette approche aborde le problème de la contamination par soi-même, où les réponses précédentes de l'assistant influencent négativement les interactions suivantes, en utilisant des références nettoyées de l'historique provenant de la politique du modèle lui-même.

Researchers have introduced MAIGO, an innovative on-policy self-distillation method designed to mitigate the lost-in-conversation (LiC) gap in large language models (LLMs). This approach addresses the issue of self-contamination, where previous assistant replies negatively influence subsequent interactions, by utilizing history-cleaned references from the model's own policy.

MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation

arXiv:2603.24481v2 Announce Type: replace-cross 
Abstract: Miscalibrated confidence scores are a practical obstacle to deploying AI in clinical settings. A model that is always overconfident offers no useful signal for deferral. We present a multi-agent framework that combines domain-specific specialist agents with Two-Phase Verification (Wu et al., 2024) and S-Score Weighted Fusion to improve both calibration and discrimination in medical multiple-choice question answering. Four specialist agents (respiratory, cardiology, neurology, gastroenterology) generate independent diagnoses using Qwen2.5-7B-Instruct. Each diagnosis undergoes a two-phase self-verification process that measures internal consistency and produces a Specialist Confidence Score (S-score). The S-scores drive a weighted fusion strategy that selects the final answer and calibrates the reported confidence. We evaluate on high-disagreement subsets of MedQA-USMLE and MedMCQA (100 and 250 questions). All results are specific to this filtered regime. On MedQA-250, the full system achieves ECE = 0.091 (74.4% reduction over the single-specialist baseline) and AUROC = 0.630 (+0.056) at 59.2% accuracy. Calibration gains of 49-74% hold across all four settings. Ablation analysis reveals that Two-Phase Verification drives ECE reduction while multi-agent reasoning drives AUROC improvement, suggesting that consistency checking and ensemble aggregation address different failure modes of LLM uncertainty. Whether the resulting confidence signal is sufficient to support clinical deferral decisions in practice remains a direction for future investigation.

تم تطوير إطار عمل متعدد الوكلاء لتحسين معايرة عدم اليقين في أسئلة الاختيار المتعدد الطبية (MCQA)، حيث يعالج مشكلة درجات الثقة غير المعايرة التي تعيق نشر الذكاء الاصطناعي في البيئات السريرية. يستخدم هذا الإطار أربعة وكلاء متخصصين في التنفس، وأمراض القلب، والأعصاب، وأمراض الجهاز الهضمي، الذين يولدون تشخيصات مستقلة ويخضعون لعملية تحقق ذاتي من مرحلتين لإنتاج درجات ثقة متخصصة (S-scores).

Se ha desarrollado un nuevo marco multiagente para mejorar la calibración de la incertidumbre en las preguntas de opción múltiple médicas (MCQA), abordando el problema de los puntajes de confianza mal calibrados que obstaculizan el despliegue de la IA en entornos clínicos. Este marco utiliza cuatro agentes especialistas en respiración, cardiología, neurología y gastroenterología, que generan diagnósticos independientes y pasan por un proceso de auto-verificación en dos fases para producir Puntajes de Confianza Especializados (S-scores).

Un nouveau cadre multi-agents a été développé pour améliorer la calibration de l'incertitude dans les questions à choix multiples médicales (MCQA), abordant le problème des scores de confiance mal calibrés qui entravent le déploiement de l'IA dans les milieux cliniques. Ce cadre utilise quatre agents spécialisés en pneumologie, cardiologie, neurologie et gastro-entérologie, qui génèrent des diagnostics indépendants et subissent un processus de vérification en deux phases pour produire des scores de confiance spécialisés (S-scores).

A new multi-agent framework has been developed to enhance uncertainty calibration in medical multiple-choice question answering (MCQA), addressing the issue of miscalibrated confidence scores that hinder AI deployment in clinical settings. This framework utilizes four specialist agents in respiratory, cardiology, neurology, and gastroenterology, which generate independent diagnoses and undergo a two-phase self-verification process to produce Specialist Confidence Scores (S-scores).

MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation

Was this article worth reading? Share it

Ready to build your own newsroom?