arXiv:2605.28818v1 Announce Type: new 
Abstract: Large language models (LLMs) have become increasingly useful computational models of human language processing, but it remains unclear whether vision-language learning makes text representations more human-like during natural reading. Here, we address this question by comparing tightly matched LLM and vision-language model (VLM) pairs under a strictly text-only setting, allowing us to isolate the effect of multimodal training history from online visual input or cross-modal fusion. We evaluate model alignment with a human natural-reading dataset that includes whole-cortex fMRI responses and synchronized eye-tracking saccades. Our findings demonstrate that multimodal pretraining may not confer a uniform, global advantage in human alignment during natural reading, indicating that language-internal representations remain the key factor for modeling human text processing. However, the VLM advantage could emerge more selectively when sentences contain stronger visual semantic content, with converging evidence from both fMRI and eye-movement alignments. Together, our findings provide a controlled in silico framework for testing how visual learning history shapes model-human alignment of language processing, suggesting that multimodal pretraining contributes selectively rather than globally to human-like language representations during natural reading.

دراسة حديثة نُشرت على arXiv تبحث في فعالية نماذج اللغة المرئية (VLM) مقارنةً بنماذج اللغة الكبيرة (LLM) في تعزيز التوافق البشري أثناء القراءة الطبيعية. تستخدم الدراسة إعدادًا نصيًا فقط لعزل تأثيرات التدريب متعدد الوسائط، مما يكشف أن VLM قد لا توفر ميزة متسقة على LLM في هذا السياق.

Un estudio reciente publicado en arXiv investiga la efectividad de los modelos de lenguaje visual (VLM) en comparación con los modelos de lenguaje grande (LLM) para mejorar la alineación humana durante la lectura natural. La investigación utiliza un entorno solo de texto para aislar los efectos del entrenamiento multimodal, revelando que los VLM pueden no proporcionar una ventaja consistente sobre los LLM en este contexto.

Une étude récente publiée sur arXiv examine l'efficacité des modèles de langage visuel (VLM) par rapport aux modèles de langage large (LLM) pour améliorer l'alignement humain lors de la lecture naturelle. La recherche utilise un cadre uniquement textuel pour isoler les effets de l'entraînement multimodal, révélant que les VLM ne fournissent pas d'avantage constant par rapport aux LLM dans ce contexte.

A recent study published on arXiv investigates the effectiveness of vision-language models (VLMs) compared to large language models (LLMs) in enhancing human alignment during natural reading. The research utilizes a text-only setting to isolate the effects of multimodal training, revealing that VLMs may not provide a consistent advantage over LLMs in this context.