arXiv:2511.08003v2 Announce Type: replace 
Abstract: Current Video Large Language Models (VideoLLMs) suffer from quadratic computational complexity and key-value cache scaling, due to their reliance on processing excessive redundant visual tokens. To address this problem, we propose SharpV, a minimalist and efficient method for adaptive pruning of visual tokens and KV cache. Different from most uniform compression approaches, SharpV dynamically adjusts pruning ratios based on spatial-temporal information. Remarkably, this adaptive mechanism occasionally achieves performance gains over dense models, offering a novel paradigm for adaptive pruning. During the KV cache pruning stage, based on observations of visual information degradation, SharpV prunes degraded visual features via a self-calibration manner, guided by similarity to original visual features. In this way, SharpV achieves hierarchical cache pruning from the perspective of information bottleneck, offering a new insight into VideoLLMs' information flow. Experiments on multiple public benchmarks demonstrate the superiority of SharpV. Moreover, to the best of our knowledge, SharpV is notably the first two-stage pruning framework that operates without requiring access to exposed attention scores, ensuring full compatibility with hardware acceleration techniques like Flash Attention.

تم اقتراح طريقة جديدة تُدعى SharpV لتحسين كفاءة نماذج اللغة الكبيرة للفيديو (VideoLLMs) من خلال تقليم توكنات بصرية وذاكرات مفتاح-قيمة (KV) بشكل تكيفي، مما يعالج التعقيد الحسابي المرتبط بالبيانات البصرية الزائدة. هذه الطريقة تعدل نسب التقليم ديناميكيًا بناءً على المعلومات الزمانية والمكانية، بهدف تحسين الأداء مع تقليل استهلاك الموارد.

Se ha propuesto un nuevo método llamado SharpV para mejorar la eficiencia de los Modelos de Lenguaje de Video (VideoLLMs) mediante la poda adaptativa de tokens visuales y cachés de clave-valor (KV), abordando la complejidad computacional asociada con datos visuales redundantes. Este enfoque ajusta dinámicamente las proporciones de poda en función de la información espacio-temporal, con el objetivo de mejorar el rendimiento mientras reduce el consumo de recursos.

Une nouvelle méthode nommée SharpV a été proposée pour améliorer l'efficacité des modèles de langage vidéo (VideoLLMs) en élaguant de manière adaptative les jetons visuels et les caches de clés-valeurs (KV), abordant la complexité computationnelle associée aux données visuelles redondantes. Cette approche ajuste dynamiquement les ratios d'élagage en fonction des informations spatio-temporelles, visant à améliorer les performances tout en réduisant la consommation de ressources.

A new method named SharpV has been proposed to enhance the efficiency of Video Large Language Models (VideoLLMs) by adaptively pruning visual tokens and key-value (KV) caches, addressing the computational complexity associated with redundant visual data. This approach dynamically adjusts pruning ratios based on spatial-temporal information, aiming to improve performance while reducing resource consumption.

Sharp Eyes and Memory for VideoLLMs: Information-Aware Visual Token Pruning for Efficient and Reliable VideoLLM Reasoning

One More Thing in AI – Your Shortcut to AI Mastery

Sharp Eyes and Memory for VideoLLMs: Information-Aware Visual Token Pruning for Efficient and Reliable VideoLLM Reasoning

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

VideoDigest

Videotok

Videolulu

LangWatch

Ready to build your own newsroom?