arXiv:2603.02097v5 Announce Type: replace 
Abstract: Open-ended medical LLM evaluation remains weakly grounded in physician-calibrated coverage of clinically relevant response criteria, especially in localized clinical settings. We introduce \textsc{ClinConsensus}, a Chinese medical benchmark of 2{,}500 expert-curated cases spanning 36 specialties, 12 task themes, multiple difficulty levels, and lay-facing versus professional-facing settings. Each case is paired with 30 case-specific binary rubric criteria. To evaluate whether responses satisfy enough physician-authored criteria, we propose \emph{Clinician-Anchored Coverage Score} (CACS), a physician-calibrated threshold metric instantiated at \(k=10\), and develop a dual-judge framework combining a GPT-5.1 grader with a physician-supervised Qwen3-8B judge. Evaluating 11 frontier LLMs, we find a persistent coverage gap: Rubric Accuracy ranges from 39.6\% to 52.1\%, whereas CACS@10 ranges from 17.8\% to 32.9\%, leaving a 19.2--21.9 point gap across models. Stratified analyses further reveal substantial variation across reasoning, evidence use, structured extraction, medication instructions, follow-up, and dialogue register. These results suggest that medical LLM evaluation should measure thresholded, rubric-grounded clinical coverage rather than average partial correctness.

يمثل تقديم ClinConsensus تقدمًا كبيرًا في تقييم تغطية المعايير السريرية في نماذج اللغة الكبيرة الطبية (LLMs) الصينية، حيث يتضمن 2500 حالة تم اختيارها من قبل خبراء عبر 36 تخصصًا وإطار عمل مزدوج القضاة يجمع بين مُقيِّم GPT-5.1 وقاضٍ Qwen3-8B تحت إشراف طبيب.

La introducción de ClinConsensus marca un avance significativo en la evaluación de la cobertura de rúbricas clínicas en modelos de lenguaje médico (LLMs) chinos, presentando 2,500 casos seleccionados por expertos en 36 especialidades y un marco de doble juez que combina un evaluador GPT-5.1 con un juez Qwen3-8B supervisado por un médico.

L'introduction de ClinConsensus représente une avancée significative dans l'évaluation de la couverture des rubriques cliniques dans les modèles de langage médical (LLMs) chinois, avec 2 500 cas sélectionnés par des experts couvrant 36 spécialités et un cadre à double juge combinant un évaluateur GPT-5.1 avec un juge Qwen3-8B supervisé par un médecin.

The introduction of ClinConsensus marks a significant advancement in the evaluation of clinical rubric coverage in Chinese medical large language models (LLMs), featuring 2,500 expert-curated cases across 36 specialties and a dual-judge framework that combines a GPT-5.1 grader with a physician-supervised Qwen3-8B judge.