arXiv:2510.27254v1 Announce Type: cross 
Abstract: Instruction-tuned Large Language Models (LLMs) underperform on low resource, non-Latin scripts due to tokenizer fragmentation and weak cross-lingual coupling. We present LLINK (Latent Language Injection for Non-English Knowledge), a compute efficient language-as-modality method that conditions an instruction-tuned decoder without changing the tokenizer or retraining the decoder. First, we align sentence embeddings from a frozen multilingual encoder to the decoder's latent embedding space at a reserved position via a lightweight contrastive projector. Second, the vector is expanded into K soft slots and trained with minimal adapters so the frozen decoder consumes the signal. LLINK substantially improves bilingual retrieval and achieves 81.3% preference over the base model and 63.6% over direct fine-tuning in LLM-judged Q&A evaluations. We further find that improvements can be attributed to reduced tokenization inflation and a stronger cross lingual alignment, despite the model having residual weaknesses in numeric fidelity. Treating low resource languages as a modality offers a practical path to stronger cross-lingual alignment in lightweight LLMs.

تُحدث مقاربة جديدة تُدعى LLINK (حقن اللغة الكامنة للمعرفة غير الإنجليزية) ضجة في مجال نماذج اللغة. تتناول التحديات التي تواجه نماذج اللغة الكبيرة عند التعامل مع النصوص غير اللاتينية ذات الموارد المحدودة. من خلال محاذاة تمثيلات الجمل دون الحاجة إلى إعادة التدريب أو تغيير المُرمز، تُحسن LLINK الأداء عبر اللغات. هذه الابتكار مهم لأنه يفتح الأبواب لفهم ومعالجة لغات متنوعة بشكل أفضل، مما يجعل التكنولوجيا أكثر وصولاً للناطقين بغير الإنجليزية.

Un nuevo enfoque llamado LLINK (Inyección Latente de Lenguaje para Conocimiento No Inglés) está causando revuelo en el campo de los modelos de lenguaje. Aborda los desafíos que enfrentan los Modelos de Lenguaje de Gran Tamaño (LLMs) al tratar con scripts no latinos de bajos recursos. Al alinear los embeddings de oraciones sin necesidad de reentrenar o cambiar el tokenizador, LLINK mejora el rendimiento entre lenguas. Esta innovación es significativa ya que abre puertas para una mejor comprensión y procesamiento de diversos idiomas, haciendo que la tecnología sea más accesible para los hablantes no ingleses.

Une nouvelle approche appelée LLINK (Injection Latente de Langue pour la Connaissance Non-Anglaise) fait sensation dans le domaine des modèles linguistiques. Elle s'attaque aux défis rencontrés par les modèles de langage de grande taille (LLMs) lorsqu'ils traitent des scripts non latins à faibles ressources. En alignant les embeddings de phrases sans avoir besoin de réentraînement ou de modification du tokenizer, LLINK améliore les performances interlinguales. Cette innovation est importante car elle ouvre des portes pour une meilleure compréhension et un meilleur traitement des langues diverses, rendant finalement la technologie plus accessible aux non-anglophones.

A new approach called LLINK (Latent Language Injection for Non-English Knowledge) is making waves in the field of language models. It addresses the challenges faced by instruction-tuned Large Language Models (LLMs) when dealing with low-resource, non-Latin scripts. By aligning sentence embeddings without the need for retraining or changing the tokenizer, LLINK enhances cross-lingual performance. This innovation is significant as it opens doors for better understanding and processing of diverse languages, ultimately making technology more accessible to non-English speakers.

Languages are Modalities: Cross-Lingual Alignment via Encoder Injection

arXiv:2511.08066v5 Announce Type: replace-cross 
Abstract: Recent years have witnessed the rapid advancements of large language models (LLMs) and their expanding applications, leading to soaring demands for computational resources. The widespread adoption of test-time scaling further intensifies the tension between model capability and resource consumption, highlighting the importance of inference efficiency. However, a unified metric that accurately reflects an LLM's efficiency across diverse model sizes and architectures remains absent. Motivated by the correlation between compression and intelligence, we introduce information capacity, a measure of model efficiency based on text compression performance relative to computational complexity. A distinctive feature of information capacity is its incorporation of tokenizer efficiency, which affects inference costs but is often neglected in LLM evaluations. We assess the information capacity of 52 open-source models and observe a consistent information capacity among different-sized models within a series. Experiments on 5 heterogeneous datasets reveal strong linguistic bias in mainstream LLMs. Three major factors of information capacity include tokenizer efficiency, pretraining data, and the mixture-of-experts architecture. Empirical results verify the accuracy of performance prediction across model sizes based on information capacity and show the correlation between information capacity and benchmark scores.

قدمت دراسة حديثة مفهوم سعة المعلومات لتقييم كفاءة نماذج اللغة الكبيرة (LLMs) من خلال أداء ضغط النص بالنسبة إلى التعقيد الحسابي. تهدف هذه المقياس إلى معالجة الطلب المتزايد على الموارد الحاسوبية مع تزايد اعتماد نماذج اللغة الكبيرة، مما يبرز أهمية كفاءة الاستدلال.

Un estudio reciente presenta el concepto de capacidad de información para evaluar la eficiencia de los grandes modelos de lenguaje (LLMs) a través del rendimiento de compresión de texto en relación con la complejidad computacional. Esta métrica busca abordar la creciente demanda de recursos computacionales a medida que los LLMs se adoptan más ampliamente, destacando la importancia de la eficiencia en la inferencia.

Une étude récente introduit le concept de capacité d'information pour évaluer l'efficacité des grands modèles de langage (LLMs) à travers la performance de compression de texte par rapport à la complexité computationnelle. Cette métrique vise à répondre à la demande croissante de ressources computationnelles alors que les LLMs deviennent plus largement adoptés, soulignant l'importance de l'efficacité d'inférence.

A recent study introduces the concept of information capacity to evaluate the efficiency of large language models (LLMs) through text compression performance relative to computational complexity. This metric aims to address the growing demand for computational resources as LLMs become more widely adopted, highlighting the importance of inference efficiency.

Languages are Modalities: Cross-Lingual Alignment via Encoder Injection

Was this article worth reading? Share it

LucidQuery AI

OpenL Translator

VideoDubber Video Translator

Dubsmart LLC

Llanai

Langfuse

Ready to build your own newsroom?