Procedure Learning via Regularized Gromov-Wasserstein Optimal Transport

The recent study on self-supervised procedure learning presents a novel framework that leverages fused Gromov-Wasserstein optimal transport to effectively identify key steps and their sequences from unlabeled videos. Traditional methods often struggle with variations in order, background noise, and repeated actions, which can hinder their performance. By addressing these issues, the proposed framework integrates contrastive regularization to avoid degenerate solutions that map all frames to a single cluster. This innovation not only enhances temporal alignment but also demonstrates superior performance in extensive experiments conducted on egocentric and third-person benchmarks, outperforming previous approaches, including OPEL, which relied on classical Kantorovich optimal transport. The findings underscore the potential of this new method in advancing the field of video analysis and procedure learning, marking a significant step forward in the use of self-supervised learning techniqu…