arXiv:2601.09853v3 Announce Type: replace 
Abstract: Real-world health questions from patients often unintentionally embed false assumptions or premises. In such cases, safe medical communication typically involves redirection: addressing the implicit misconception and then responding to the underlying patient context, rather than the original question. While large language models (LLMs) are increasingly being used by lay users for medical advice, they have not yet been tested for this crucial competency. Therefore, in this work, we investigate how LLMs react to false premises embedded within real-world health questions. We develop a semi-automated pipeline to curate MedRedFlag, a dataset of 1100+ questions sourced from Reddit that require redirection. We then systematically compare responses from state-of-the-art LLMs to those from clinicians. Our analysis reveals that LLMs often fail to redirect problematic questions, even when the problematic premise is detected, and provide answers that could lead to suboptimal medical decision making. Our benchmark and results reveal a novel and substantial gap in how LLMs perform under the conditions of real-world health communication, highlighting critical safety concerns for patient-facing medical AI systems. Code and dataset are available at https://github.com/srsambara-1/MedRedFlag.

كشفت دراسة حديثة حول نماذج اللغة الكبيرة (LLMs) عن قصورها في معالجة المفاهيم الخاطئة المضمنة في الأسئلة الصحية الواقعية. استخدمت الدراسة مجموعة بيانات تضم أكثر من 1,100 سؤال من Reddit، ووجدت أن نماذج LLMs غالبًا ما تفشل في إعادة توجيه الاستفسارات الإشكالية، حتى عند تحديد المفاهيم الخاطئة الأساسية.

Una reciente investigación sobre modelos de lenguaje de gran tamaño (LLMs) ha revelado sus deficiencias para abordar las ideas erróneas incrustadas en preguntas de salud del mundo real. El estudio, que utilizó un conjunto de datos de más de 1,100 preguntas de Reddit, encontró que los LLMs a menudo fallaban en redirigir consultas problemáticas, incluso cuando se identificaban las premisas erróneas subyacentes.

Une récente enquête sur les modèles de langage de grande taille (LLMs) a révélé leurs lacunes dans la gestion des idées fausses intégrées dans les questions de santé du monde réel. L'étude, qui a utilisé un ensemble de données de plus de 1 100 questions provenant de Reddit, a constaté que les LLMs échouaient souvent à rediriger les demandes problématiques, même lorsque les idées fausses sous-jacentes étaient identifiées.

A recent investigation into large language models (LLMs) has revealed their shortcomings in addressing misconceptions embedded in real-world health questions. The study, which utilized a dataset of over 1,100 questions from Reddit, found that LLMs often failed to redirect problematic inquiries, even when the underlying misconceptions were identified.