FlowFeat: Pixel-Dense Embedding of Motion Profiles

The introduction of FlowFeat marks a significant advancement in computer vision, particularly in dense prediction tasks such as video object segmentation, monocular depth estimation, and semantic segmentation. Traditional networks, including transformers, often yield low-resolution feature grids that are inadequate for these applications. FlowFeat overcomes this limitation through a novel distillation technique that embeds a distribution of plausible apparent motions, leveraging optical flow networks and diverse video data. This self-supervised training framework not only approximates apparent motion effectively but also enhances the representational power of five state-of-the-art encoders. The computational efficiency and robustness of FlowFeat to inaccurate flow estimation further underscore its potential impact on the field, promising to improve the performance of various computer vision tasks significantly.