Suppressing Static Visual Cues via Normalizing Flows for Self-Supervised Video Representation Learning

TL;DR

This paper introduces a novel self-supervised video representation learning method that suppresses static visual cues using normalizing flows, leading to more generalized and less biased video features.

Contribution

It proposes a probabilistic approach to identify and suppress static cues in videos via normalizing flows, improving the robustness of learned representations.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Enhances generalization of video representations to downstream tasks.

Effectively reduces static cue bias in self-supervised learning.

Abstract

Despite the great progress in video understanding made by deep convolutional neural networks, feature representation learned by existing methods may be biased to static visual cues. To address this issue, we propose a novel method to suppress static visual cues (SSVC) based on probabilistic analysis for self-supervised video representation learning. In our method, video frames are first encoded to obtain latent variables under standard normal distribution via normalizing flows. By modelling static factors in a video as a random variable, the conditional distribution of each latent variable becomes shifted and scaled normal. Then, the less-varying latent variables along time are selected as static cues and suppressed to generate motion-preserved videos. Finally, positive pairs are constructed by motion-preserved videos for contrastive learning to alleviate the problem of representation bias to static cues. The less-biased video representation can be better generalized to various downstream tasks. Extensive experiments on publicly available benchmarks demonstrate that the proposed method outperforms the state of the art when only single RGB modality is used for pre-training.