arXiv:2510.22492v1 Announce Type: new 
Abstract: How much audio is needed to fully observe a multilingual ASR model's learned sub-token inventory across languages, and does data disparity in multilingual pre-training affect how these tokens are utilized during inference? We address this question by analyzing Whisper's decoding behavior during inference across 49 languages. By logging decoding candidate sub-tokens and tracking their cumulative discovery over time, we study the utilization pattern of the model's sub-token space. Results show that the total number of discovered tokens remains largely independent of a language's pre-training hours, indicating that data disparity does not strongly influence lexical diversity in the model's hypothesis space. Sub-token discovery rates follow a consistent exponential saturation pattern across languages, suggesting a stable time window after which additional audio yields minimal new sub-token activation. We refer to this convergence threshold as acoustic saturation time (AST). Further analyses of rank-frequency distributions reveal Zipf-like patterns better modeled by a Zipf-Mandelbrot law, and mean sub-token length shows a positive correlation with resource level. Additionally, those metrics show more favorable patterns for languages in the Latin script than those in scripts such as Cyrillic, CJK, and Semitic. Together, our study suggests that sub-token utilization during multilingual ASR inference is constrained more by the statistical, typological, and orthographic structure of the speech than by training data scale, providing an empirical basis for more equitable corpus construction and cross-lingual evaluation.

تستكشف دراسة حديثة فعالية نماذج التعرف التلقائي على الكلام (ASR) متعددة اللغات، مع التركيز بشكل خاص على أداء Whisper عبر 49 لغة. تبحث الدراسة في كمية بيانات الصوت اللازمة لاستخدام كامل لمخزون النموذج من الرموز الفرعية المتعلمة، وما إذا كانت الفجوات في البيانات أثناء التدريب المسبق تؤثر على استخدام الرموز أثناء الاستدلال. هذه التحليلات مهمة لأنها تسلط الضوء على تعقيدات أنظمة ASR متعددة اللغات وقدرتها على التكيف مع السياقات اللغوية المختلفة، وهو أمر ضروري لتحسين تقنيات الاتصال على مستوى العالم.

Un estudio reciente explora la efectividad de los modelos de Reconocimiento Automático de Voz (ASR) multilingües, centrándose específicamente en el rendimiento de Whisper en 49 idiomas. La investigación investiga cuántos datos de audio son necesarios para utilizar completamente el inventario de sub-tokens aprendidos por el modelo y si las disparidades en los datos durante el preentrenamiento afectan el uso de tokens durante la inferencia. Este análisis es crucial, ya que arroja luz sobre las complejidades de los sistemas ASR multilingües y su capacidad para adaptarse a diversos contextos lingüísticos, lo que es esencial para mejorar las tecnologías de comunicación a nivel mundial.

Une étude récente examine l'efficacité des modèles de reconnaissance automatique de la parole (ASR) multilingues, en se concentrant spécifiquement sur les performances de Whisper dans 49 langues. La recherche analyse combien de données audio sont nécessaires pour utiliser pleinement l'inventaire de sous-tokens appris par le modèle et si les disparités de données lors de l'entraînement préalable influencent l'utilisation des tokens pendant l'inférence. Cette analyse est cruciale car elle éclaire les complexités des systèmes ASR multilingues et leur capacité à s'adapter à divers contextes linguistiques, ce qui est essentiel pour améliorer les technologies de communication à l'échelle mondiale.

A recent study explores the effectiveness of multilingual Automatic Speech Recognition (ASR) models, specifically focusing on Whisper's performance across 49 languages. The research investigates how much audio data is necessary to fully utilize the model's learned sub-token inventory and whether disparities in data during pre-training impact token usage during inference. This analysis is crucial as it sheds light on the complexities of multilingual ASR systems and their ability to adapt to varying linguistic contexts, which is essential for improving communication technologies globally.

The Limits of Data Scaling: Sub-token Utilization and Acoustic Saturation in Multilingual ASR

One More Thing in AI – Your Shortcut to AI Mastery

The Limits of Data Scaling: Sub-token Utilization and Acoustic Saturation in Multilingual ASR

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

ShareSpeak

Superwhisper

SoundWise.ai

Dubsmart LLC

Ready to build your own newsroom?