MSVQ: Self-Supervised Learning with Multiple Sample Views and Queues

TL;DR

MSVQ introduces a self-supervised learning framework that uses multiple sample views and queues with dual momentum encoders to better identify false negatives, improving visual representation learning.

Contribution

The paper proposes MSVQ, a novel self-supervised framework utilizing multiple augmented views and dual momentum encoders to enhance false negative sample detection.

Findings

Outperforms classical methods on four benchmark datasets

Effective in identifying false negatives in contrastive learning

Demonstrates high efficiency and effectiveness

Abstract

Self-supervised methods based on contrastive learning have achieved great success in unsupervised visual representation learning. However, most methods under this framework suffer from the problem of false negative samples. Inspired by the mean shift for self-supervised learning, we propose a new simple framework, namely Multiple Sample Views and Queues (MSVQ). We jointly construct three soft labels on-the-fly by utilizing two complementary and symmetric approaches: multiple augmented positive views and two momentum encoders that generate various semantic features for negative samples. Two teacher networks perform similarity relationship calculations with negative samples and then transfer this knowledge to the student network. Let the student network mimic the similarity relationships between the samples, thus giving the student network a more flexible ability to identify false negative samples in the dataset. The classification results on four benchmark image datasets demonstrate the high effectiveness and efficiency of our approach compared to some classical methods. Source code and pretrained models are available \href{https://github.com/pc-cp/MSVQ}{here}.