MMTM: Tri-Modal Topic Modeling for Long-Form Video via Similarity-Gated Fusion

The introduction of MMTM, a modular pipeline for topic discovery in long-form video, integrates speech recognition, audio and visual embeddings, and BERTopic clustering through a similarity-gated fusion approach. This method has been evaluated on German and English broadcast news, demonstrating significant improvements in topic quality metrics.

The advancements brought by MMTM are crucial for enhancing the coherence and stability of topics in long-form video content, which can lead to better viewer engagement and understanding of complex narratives.

This development reflects a growing trend in artificial intelligence where multi-modal approaches are increasingly utilized to improve topic modeling, as seen in various applications ranging from healthcare narratives to creative brainstorming, highlighting the versatility and importance of advanced topic modeling techniques across different domains.