RosettaSpeech: Zero-Shot Speech-to-Speech Translation from Monolingual Data

A recent study explores sound symbolism, revealing how Multimodal Large Language Models (MLLMs) interpret auditory information in human languages. The research introduces LEX-ICON, a dataset comprising 8,052 words and 2,930 pseudo-words across four languages, examining MLLMs' phonetic iconicity through phoneme-level attention scores.

LongCat-Image has been introduced as an innovative open-source bilingual foundation model for image generation, specifically designed to enhance multilingual text rendering and photorealism. This model employs advanced data curation strategies throughout its training phases, achieving state-of-the-art performance in text-rendering and aesthetic quality, particularly for complex Chinese characters.

SwissGov-RSD has been introduced as the first naturalistic, document-level, cross-lingual dataset designed for recognizing semantic differences across documents in multiple languages, including English, German, French, and Italian. This dataset includes 224 multi-parallel documents annotated at the token level by human annotators, addressing a previously underexplored area in text generation evaluation and multilingual content alignment.

GUMBridge has been introduced as a new resource for bridging anaphora, encompassing 16 diverse genres of English. This corpus aims to provide comprehensive coverage of the phenomenon, which involves understanding references in discourse that depend on previous entities, such as identifying 'the door' as belonging to 'a house.'

A new benchmark corpus for Telugu-English speech translation, named TeluguST-46, has been developed, comprising 46 hours of manually verified data. This initiative addresses the underexplored area of speech translation for Telugu, a language spoken by over 80 million people, and includes a systematic evaluation of various translation architectures, highlighting the performance of IndicWhisper + IndicMT and finetuned SeamlessM4T models.

Structure-inducing Language Models (SiLM) have been trained from scratch using three different architectures: Structformer, UDGN, and GPST, focusing on their syntactic generalization capabilities and performance across various NLP tasks. The study evaluates the models on their induced syntactic representations, grammaticality judgment tasks, and training dynamics, revealing no single architecture excels across all metrics.

The TRepLiNa method, which combines Centered Kernel Alignment (CKA) and REPINA, has been introduced to enhance low-resource machine translation, particularly for Indian languages like Mundari, Santali, and Bhili, using the Aya-23 8B model. This approach aims to improve translation quality from low-resource languages to high-resource languages such as Hindi and English.