arXiv:2605.26601v1 Announce Type: new 
Abstract: Vision-language models have progressed rapidly, but Tibetan remains a severely underserved low-resource language due to the lack of reproducible training and evaluation infrastructure. To fill this gap, we introduce FTibSuite, a comprehensive resource suite for Tibetan vision-language research, consisting of FTibData (human-verified multimodal training corpora spanning continual pretraining, image-text alignment, and instruction tuning data), FTibBench (Tibetan adaptations of five mainstream multimodal benchmarks with a hierarchical quality-control workflow to reduce translation noise), and FTibVLM, a reproducible baseline built on Qwen3-VL-8B-Instruct via a three-stage adaptation pipeline. Experiments on FTibBench show FTibVLM delivers consistent performance gains across all tasks, such as improving MMBench accuracy from 42.97 to 67.78 and POPE-random accuracy from 47.53 to 80.56, while retaining the backbone's original Chinese capabilities with minimal degradation, providing the first standardized foundation for Tibetan multimodal research.

يمثل تقديم FTibSuite تقدمًا كبيرًا في نمذجة اللغة البصرية-التبتية، حيث يوفر موارد أساسية مثل FTibData وFTibBench وFTibVLM لمعالجة التحديات التي تواجه هذه اللغة ذات الموارد المحدودة. تهدف هذه المجموعة الشاملة إلى تحسين بنية التدريب والتقييم، التي كانت تفتقر إليها اللغة التبتية.

La introducción de FTibSuite marca un avance significativo en la modelización de lenguaje visual-tibetano, proporcionando recursos esenciales como FTibData, FTibBench y FTibVLM para abordar los desafíos que enfrenta este idioma de bajos recursos. Esta suite integral tiene como objetivo mejorar la infraestructura de entrenamiento y evaluación, que ha sido deficiente para el tibetano.

L'introduction de FTibSuite représente une avancée significative dans la modélisation du langage visuel-tibétain, fournissant des ressources essentielles telles que FTibData, FTibBench et FTibVLM pour relever les défis rencontrés par cette langue à faibles ressources. Cette suite complète vise à améliorer l'infrastructure d'entraînement et d'évaluation, qui a fait défaut pour le tibétain.

The introduction of FTibSuite marks a significant advancement in Tibetan vision-language modeling, providing essential resources such as FTibData, FTibBench, and FTibVLM to address the challenges faced by this low-resource language. This comprehensive suite aims to enhance training and evaluation infrastructure, which has been lacking for Tibetan.

FTibSuite: A Comprehensive Resource Suite for Tibetan Vision-Language Modeling

arXiv:2606.07647v1 Announce Type: new 
Abstract: Large vision language models (LVLMs) have made rapid advancements and are deployed across various applications, yet hallucinations remain a major challenge. Activation steering is appealing due to its minimal training overhead and controllability at inference time. However, we found that during autoregressive decoding, visual conditioning affects token prediction sparsely and locally across decoding steps, and many existing methods that average image-versus-no-image differences over the entire sequence dilute these critical signals, yielding low signal-to-noise ratio steering directions. Additionally, many existing methods apply a fixed steering strength, which misallocates the intervention budget, over-perturbs non-critical tokens, and can cause instability. To address these limitations, we propose Token-Level Visual-Sensitivity Steering (TLVS) for hallucination mitigation. Our approach first extracts token-level steering vectors and refines them, and then applies fine-grained, visual-sensitivity-adaptive steering only where it matters. This lightweight, plug-and-play mechanism requires only minimal training for calibration and can be applied across diverse vision-language models. It modulates the steering strength at each decoding step, selectively suppressing hallucination-prone spans while preserving evidence-grounded content. We evaluate TLVS on several benchmarks, including POPE, AMBER, CHAIR (COCO), MMHal, and HallusionBench, demonstrating consistent improvements over previous steering methods.

أدت التطورات الأخيرة في نماذج اللغة البصرية الكبيرة (LVLMs) إلى تقديم توجيه الحساسية البصرية على مستوى الرموز (TLVS)، وهو نهج مبتكر يهدف إلى التخفيف من الهلوسات أثناء فك التشفير التلقائي. تتناول هذه الطريقة قيود الأساليب الحالية من خلال استخراج متجهات توجيه على مستوى الرموز، مما يعزز قدرة النموذج على التنبؤ بدقة مع تقليل عبء التدريب.

Los recientes avances en los Modelos de Lenguaje Visual Grande (LVLM) han llevado a la introducción de la Dirección de Sensibilidad Visual a Nivel de Token (TLVS), un enfoque novedoso destinado a mitigar las alucinaciones durante la decodificación autorregresiva. Este método aborda las limitaciones de las técnicas existentes al extraer vectores de dirección a nivel de token, mejorando la capacidad del modelo para predecir con precisión mientras minimiza la sobrecarga de entrenamiento.

Des avancées récentes dans les grands modèles de langage visuel (LVLM) ont conduit à l'introduction de la direction de sensibilité visuelle au niveau des jetons (TLVS), une approche novatrice visant à atténuer les hallucinations lors du décodage autorégressif. Cette méthode aborde les limitations des techniques existantes en extrayant des vecteurs de direction au niveau des jetons, améliorant ainsi la capacité du modèle à prédire avec précision tout en minimisant les coûts d'entraînement.

Recent advancements in Large Vision Language Models (LVLMs) have led to the introduction of Token-Level Visual-Sensitivity Steering (TLVS), a novel approach aimed at mitigating hallucinations during autoregressive decoding. This method addresses the limitations of existing techniques by extracting token-level steering vectors, enhancing the model's ability to predict accurately while minimizing training overhead.

FTibSuite: A Comprehensive Resource Suite for Tibetan Vision-Language Modeling

Was this article worth reading? Share it

Ready to build your own newsroom?