Quality Over Quantity? LLM-Based Curation for a Data-Efficient Audio-Video Foundation Model

The recent development of the Audio-Video Vector Alignment (AVVA) framework marks a significant advancement in the integration of audio and visual data for training multimodal foundational models. By focusing on scene alignment rather than just temporal synchronization, AVVA enhances the efficiency of data curation using Large Language Models (LLMs). This innovation not only streamlines the selection of aligned training data segments but also incorporates the Whisper model, which is pivotal for speech recognition. This progress is crucial as it paves the way for more effective and data-efficient models in the audio-visual domain.