arXiv:2505.14160v4 Announce Type: replace-cross 
Abstract: Multilingual vision-language models (VLMs) promise universal image-text retrieval, yet their social biases remain underexplored. We perform the first systematic audit of four public multilingual CLIP variants: M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and the debiased SigLIP-2, covering ten languages that differ in resource availability and morphological gender marking. Using balanced subsets of FairFace and the PATA stereotype suite in a zero-shot setting, we quantify race and gender bias and measure stereotype amplification. Contrary to the intuition that multilinguality mitigates bias, every model exhibits stronger gender skew than its English-only baseline. CAPIVARA-CLIP shows its largest biases precisely in the low-resource languages it targets, while the shared encoder of NLLB-CLIP and SigLIP-2 transfers English gender stereotypes into gender-neutral languages; loosely coupled encoders largely avoid this leakage. Although SigLIP-2 reduces agency and communion skews, it inherits -- and in caption-sparse contexts (e.g., Xhosa) amplifies -- the English anchor's crime associations. Highly gendered languages consistently magnify all bias types, yet gender-neutral languages remain vulnerable whenever cross-lingual weight sharing imports foreign stereotypes. Aggregated metrics thus mask language-specific hot spots, underscoring the need for fine-grained, language-aware bias evaluation in future multilingual VLM research.

تكشف دراسة حول نماذج الرؤية-اللغة متعددة اللغات (VLM) عن تحيزات كبيرة في الجنس والعرق، مما يتعارض مع الافتراض القائل بأن التعدد اللغوي يقلل من التحيز. يُظهر تدقيق أربعة نماذج—M-CLIP وNLLB-CLIP وCAPIVARA-CLIP وSigLIP-2—أن جميعها تظهر انحيازًا أكبر للجنس مقارنة بالنماذج التي تقتصر على اللغة الإنجليزية. ومن الجدير بالذكر أن CAPIVARA-CLIP تُظهر تحيزات أكبر في اللغات ذات الموارد المنخفضة، بينما تنقل NLLB-CLIP وSigLIP-2 الصور النمطية للجنس الإنجليزي إلى لغات محايدة.

Un estudio sobre modelos de visión-lenguaje multilingües (VLM) revela sesgos significativos de género y raza, contradiciendo la suposición de que el multilingüismo reduce los sesgos. La auditoría de cuatro modelos—M-CLIP, NLLB-CLIP, CAPIVARA-CLIP y SigLIP-2—demuestra que todos exhiben un sesgo de género más fuerte en comparación con los modelos solo en inglés. Notablemente, CAPIVARA-CLIP muestra sesgos aumentados en lenguas de bajos recursos, mientras que NLLB-CLIP y SigLIP-2 transfieren estereotipos de género ingleses a lenguas neutrales.

Une étude sur les modèles de vision-langage multilingues (VLM) révèle des biais de genre et raciaux significatifs, contredisant l'hypothèse selon laquelle la multilinguisme réduit les biais. L'audit de quatre modèles—M-CLIP, NLLB-CLIP, CAPIVARA-CLIP et SigLIP-2—démontre que tous présentent un biais de genre plus fort par rapport aux modèles uniquement en anglais. Notamment, CAPIVARA-CLIP montre des biais accrus dans les langues à faibles ressources, tandis que NLLB-CLIP et SigLIP-2 transfèrent les stéréotypes de genre anglais dans des langues neutres.

A study on multilingual vision-language models (VLMs) reveals significant gender and racial biases, contradicting the assumption that multilinguality reduces bias. The audit of four models—M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and SigLIP-2—demonstrates that all exhibit stronger gender skew compared to English-only models. Notably, CAPIVARA-CLIP shows heightened biases in low-resource languages, while NLLB-CLIP and SigLIP-2 transfer English stereotypes into gender-neutral languages.

Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models

Was this article worth reading? Share it

LucidQuery AI

ShareSpeak

SwapAnything.io

OpenL Translator

Dubsmart LLC

PrettyPolly

Ready to build your own newsroom?