A Generalized Learning Framework for Self-Supervised Contrastive Learning

TL;DR

This paper introduces a unified framework for self-supervised contrastive learning, analyzing existing methods and proposing a new adaptive calibration technique to improve feature space class separation without labels.

Contribution

It unifies existing SSCL methods under a generalized framework and proposes ADC, a novel method to enhance intra-class compactness and inter-class separability.

Findings

ADC improves feature space class separation.

Theoretical analysis supports ADC's effectiveness.

Empirical results show ADC outperforms existing methods.

Abstract

Self-supervised contrastive learning (SSCL) has recently demonstrated superiority in multiple downstream tasks. In this paper, we generalize the standard SSCL methods to a Generalized Learning Framework (GLF) consisting of two parts: the aligning part and the constraining part. We analyze three existing SSCL methods: BYOL, Barlow Twins, and SwAV, and show that they can be unified under GLF with different choices of the constraining part. We further propose empirical and theoretical analyses providing two insights into designing the constraining part of GLF: intra-class compactness and inter-class separability, which measure how well the feature space preserves the class information of the inputs. However, since SSCL can not use labels, it is challenging to design a constraining part that satisfies these properties. To address this issue, we consider inducing intra-class compactness and inter-class separability by iteratively capturing the dynamic relationship between anchor and other samples and propose a plug-and-play method called Adaptive Distribution Calibration (ADC) to ensure that samples that are near or far from the anchor point in the original input space are closer or further away from the anchor point in the feature space. Both the theoretical analysis and the empirical evaluation demonstrate the superiority of ADC.