arXiv:2605.23651v2 Announce Type: replace 
Abstract: While factual correctness and task-performance have been in focus of Large Language Model (LLM) research for a long time, the fundamental question of how human-like generated texts are on a linguistic level has been underexplored. From a corpus-linguistic perspective, language production is inherently context-dependent, with distinct communicative contexts giving rise to differences in frequencies and co-occurrence patterns of linguistic features. A text failing to adhere to these patterns can be content-wise correct, but still be unfavorable to human readers. In this work, we propose a context-aware evaluation framework in which human-likeness is assessed using a two-sample problem between the linguistic feature distribution of a human reference corpus for a given register and a corresponding LLM-generated corpus. We implement this framework using the Maximum Mean Discrepancy (MMD) and the 67 lexico-grammatical features introduced by Biber, which are commonly applied in corpus linguistics. In our experiments, we compare seven instruction-tuned, open-source models across five English-language datasets spanning distinct registers against a human baseline. While across all tested setups, LLMs deviate from the human baseline, which models are closest to human language depends on the register and is not dictated by model size.

تم اقتراح إطار جديد لتقييم مدى إنسانية النصوص التي تنتجها نماذج اللغة الكبيرة (LLMs)، مع التركيز على الميزات اللغوية التي تعكس إنتاج اللغة المعتمد على السياق. يستخدم هذا الإطار مشكلة عينة ثنائية لمقارنة مخرجات LLM مع مجموعات مرجعية بشرية، بهدف تقييم الالتزام بالأنماط اللغوية التي تميز التواصل البشري.

Se ha propuesto un nuevo marco para evaluar la humanidad de los textos generados por los Modelos de Lenguaje Grande (LLMs), centrándose en las características lingüísticas que reflejan la producción de lenguaje dependiente del contexto. Este marco utiliza un problema de dos muestras para comparar las salidas de los LLM con los corpus de referencia humanos, con el objetivo de evaluar la adherencia a los patrones lingüísticos que caracterizan la comunicación humana.

Un nouveau cadre d'évaluation de l'humanité des textes générés par les grands modèles de langage (LLMs) a été proposé, mettant l'accent sur les caractéristiques linguistiques qui reflètent la production linguistique dépendante du contexte. Ce cadre utilise un problème à deux échantillons pour comparer les sorties des LLM avec des corpus de référence humains, visant à évaluer l'adhésion aux modèles linguistiques qui caractérisent la communication humaine.

A new framework for evaluating the human-likeness of texts generated by Large Language Models (LLMs) has been proposed, focusing on the linguistic features that reflect context-dependent language production. This framework utilizes a two-sample problem to compare LLM outputs with human reference corpora, aiming to assess the adherence to linguistic patterns that characterize human communication.