arXiv:2606.06788v1 Announce Type: new 
Abstract: Evaluations of large language models (LLMs) in scientific information seeking tasks have become increasingly use-centric, such as conducting live or multi-turn evaluations with real users. These evaluations still assume a single, static chat interface, but as models are integrated into new interfaces, evaluations must shift to incorporate interface-specific criteria. We propose a new evaluation framework based on a formative study with $16$ participants that tests models' ability to generate multiple responses to one query that differ along an interpretable axis of language (language complexity), inspired by direct manipulation interfaces from human-centered design literature. We evaluate GPT-5.1, GPT-5 mini, Claude Sonnet 4.5 + Thinking, and DeepSeek-V3.1 by generating 5 responses at different levels of language complexity for $98$ scientific queries. While models vary complexity across responses, most changes remain inconsistent, with the best performing model (Claude Sonnet 4.5) only shifting reliable complexity measures in the correct direction $46\%$ of the time. Our findings hold with increased sample size and alternative complexity levels.

أجرى دراسة حديثة تقييمًا للإمكانات التفاعلية لنماذج اللغة الكبيرة (LLMs) من خلال اختبار قدرتها على توليد استجابات متعددة بمستويات مختلفة من تعقيد اللغة لاستفسارات علمية. شملت الدراسة 16 مشاركًا وقيمت نماذج مثل GPT-5.1 وGPT-5 mini وClaude Sonnet 4.5 وDeepSeek-V3.1، كاشفة عن عدم اتساق في تعقيد الاستجابات.

Un estudio reciente evaluó el potencial interactivo de los grandes modelos de lenguaje (LLMs) al probar su capacidad para generar múltiples respuestas con diferentes niveles de complejidad lingüística a consultas científicas. La investigación involucró a 16 participantes y evaluó modelos como GPT-5.1, GPT-5 mini, Claude Sonnet 4.5 y DeepSeek-V3.1, revelando inconsistencias en la complejidad de las respuestas.

Une étude récente a évalué le potentiel interactif des grands modèles de langage (LLMs) en testant leur capacité à générer plusieurs réponses avec des niveaux de complexité linguistique variés à des requêtes scientifiques. La recherche a impliqué 16 participants et a évalué des modèles tels que GPT-5.1, GPT-5 mini, Claude Sonnet 4.5 et DeepSeek-V3.1, révélant des incohérences dans la complexité des réponses.

A recent study evaluated the interactive potential of large language models (LLMs) by testing their ability to generate multiple responses with varying language complexity to scientific queries. The research involved 16 participants and assessed models including GPT-5.1, GPT-5 mini, Claude Sonnet 4.5, and DeepSeek-V3.1, revealing inconsistencies in response complexity.