MoQuad: Motion-focused Quadruple Construction for Video Contrastive Learning

TL;DR

This paper introduces MoQuad, a simple yet effective contrastive learning strategy that enhances motion feature learning in videos by disturbing appearance and motion in positive and negative samples, leading to superior downstream task performance.

Contribution

MoQuad is a novel quadruple construction method that improves motion feature learning without extra auxiliary tasks or explicit temporal modeling.

Findings

Achieves 93.7% accuracy on UCF-101 after 200 epochs pre-training.

Outperforms state-of-the-art methods on downstream video tasks.

Maintains a simple contrastive learning framework without multi-task learning.

Abstract

Learning effective motion features is an essential pursuit of video representation learning. This paper presents a simple yet effective sample construction strategy to boost the learning of motion features in video contrastive learning. The proposed method, dubbed Motion-focused Quadruple Construction (MoQuad), augments the instance discrimination by meticulously disturbing the appearance and motion of both the positive and negative samples to create a quadruple for each video instance, such that the model is encouraged to exploit motion information. Unlike recent approaches that create extra auxiliary tasks for learning motion features or apply explicit temporal modelling, our method keeps the simple and clean contrastive learning paradigm (i.e.,SimCLR) without multi-task learning or extra modelling. In addition, we design two extra training strategies by analyzing initial MoQuad experiments. By simply applying MoQuad to SimCLR, extensive experiments show that we achieve superior performance on downstream tasks compared to the state of the arts. Notably, on the UCF-101 action recognition task, we achieve 93.7% accuracy after pre-training the model on Kinetics-400 for only 200 epochs, surpassing various previous methods