arXiv:2602.07026v3 Announce Type: replace 
Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of distinct modalities expressing identical semantics occupy systematically offset regions. Prior approaches to bridge this gap are largely limited by oversimplified isotropic assumptions, hindering their application in large-scale scenarios. In this paper, we address these limitations by precisely characterizing the geometric shape of the modality gap and leveraging it for efficient model scaling. First, we propose the Fixed-frame Modality Gap Theory, which decomposes the modality gap within a frozen reference frame into stable biases and anisotropic residuals. Guided by this precise modeling, we introduce ReAlign, a training-free modality alignment strategy. Utilizing statistics from massive unpaired data, ReAlign aligns text representation into the image representation distribution via a three-step process comprising Anchor, Trace, and Centroid Alignment, thereby explicitly rectifying geometric misalignment. Building on ReAlign, we propose ReVision, a scalable training paradigm for Multimodal Large Language Models~(MLLMs). ReVision integrates ReAlign into the pretraining stage, enabling the model to learn the distribution of visual representations from unpaired text before visual instruction tuning, without the need for large-scale, high-quality image-text pairs. Our framework demonstrates that statistically aligned unpaired data can effectively substitute for expensive image-text pairs, offering a robust path for the efficient scaling of MLLMs.

تم تقديم نموذج تدريب جديد لنماذج اللغة متعددة الوسائط (MLLM) يركز على معالجة الفجوة المستمرة بين الوسائط التي تتسبب في احتلال تمثيلات مختلفة لمناطق متباينة على الرغم من مشاركة دلالات متطابقة. تسمح نظرية فجوة الوسائط في إطار ثابت بتوصيف أكثر دقة لهذه الفجوة، مما يؤدي إلى تطوير ReAlign، وهي استراتيجية محاذاة بين الوسائط بدون تدريب تستخدم إحصائيات من بيانات غير متزاوجة.

Se ha introducido un nuevo paradigma de entrenamiento para los Modelos de Lenguaje Multimodal (MLLM), centrado en abordar la persistente Brecha de Modalidad que causa que las incrustaciones de diferentes modalidades ocupen regiones desfasadas a pesar de compartir semánticas idénticas. La Teoría de la Brecha de Modalidad en un marco fijo permite una caracterización más precisa de esta brecha, lo que lleva al desarrollo de ReAlign, una estrategia de alineación de modalidades sin entrenamiento que utiliza estadísticas de datos no emparejados.

Un nouveau paradigme d'entraînement pour les modèles de langage multimodaux (MLLM) a été introduit, se concentrant sur la résolution du Modality Gap persistant qui cause des embeddings de différentes modalités à occuper des régions décalées malgré le partage de sémantiques identiques. La théorie du Modality Gap dans un cadre fixe permet une caractérisation plus précise de cet écart, conduisant au développement de ReAlign, une stratégie d'alignement de modalités sans entraînement qui utilise des statistiques de données non appariées.

A new training paradigm for Multimodal Large Language Models (MLLMs) has been introduced, focusing on addressing the persistent Modality Gap that causes embeddings of different modalities to occupy offset regions despite sharing identical semantics. The proposed Fixed-frame Modality Gap Theory allows for a more precise characterization of this gap, leading to the development of ReAlign, a training-free modality alignment strategy that utilizes statistics from unpaired data.