arXiv:2605.30010v1 Announce Type: new 
Abstract: Video large language models (Video-LLMs) have demonstrated strong capabilities in video understanding tasks. However, their practical deployment is still hindered by the inefficiency introduced by processing massive amounts of visual tokens. Although recent approaches achieve extremely low token retention ratios while maintaining accuracy comparable to full-token baselines, most of them perform compression only at the late stage of prefilling, leaving the efficiency of the vision encoder unoptimized. In this paper, we first show that vision encoding contributes a large portion to the time-to-first-token (TTFT). Therefore, instead of compressing visual tokens only after the vision encoder, performing compression inside the encoder still leaves substantial room for exploration. Based on this insight, we propose EarlyTom, a training-free token compression framework that performs early-stage visual token compression inside the vision encoder, enabling significantly better TTFT reduction and higher throughput. In addition, we introduce a decoupled spatial token selection strategy that improves the overall compression effectiveness. EarlyTom reduces TTFT by up to 2.65x and FLOPs by up to 61% on a single NVIDIA A100 GPU for the LLaVA-OneVision-7B model, while maintaining accuracy comparable to the full-token baseline. These improvements substantially enhance the practicality of deploying Video-LLMs in real-world production scenarios.

تم تقديم EarlyTom كإطار عمل جديد لضغط الرموز البصرية في مرحلة مبكرة داخل نماذج اللغة الكبيرة للفيديو (Video-LLMs)، حيث يعالج عدم الكفاءة في معالجة الرموز البصرية التي تعيق النشر العملي. تعمل هذه الطريقة على تحسين مشفر الرؤية، مما يقلل بشكل كبير من الوقت حتى أول رمز (TTFT) ويعزز قدرات فهم الفيديو.

EarlyTom se ha presentado como un nuevo marco para la compresión temprana de tokens visuales dentro de los modelos de lenguaje de video (Video-LLMs), abordando las ineficiencias en el procesamiento de tokens visuales que obstaculizan el despliegue práctico. Este enfoque optimiza el codificador visual, reduciendo significativamente el tiempo hasta el primer token (TTFT) y mejorando las capacidades de comprensión de video.

EarlyTom a été introduit comme un cadre novateur pour la compression précoce des tokens visuels au sein des modèles de langage vidéo (Video-LLMs), abordant les inefficacités dans le traitement des tokens visuels qui entravent le déploiement pratique. Cette approche optimise l'encodeur visuel, réduisant significativement le temps jusqu'au premier token (TTFT) et améliorant les capacités de compréhension vidéo.

EarlyTom has been introduced as a novel framework for early-stage visual token compression within video large language models (Video-LLMs), addressing inefficiencies in processing visual tokens that hinder practical deployment. This approach optimizes the vision encoder, significantly reducing the time-to-first-token (TTFT) and enhancing video understanding capabilities.