Contrastive Learning Via Equivariant Representation

TL;DR

CLeVER introduces an equivariant contrastive learning framework that enhances robustness and efficiency by capturing augmentation-related information, leading to state-of-the-art results across various models and tasks.

Contribution

The paper proposes CLeVER, a novel equivariant contrastive learning method compatible with complex augmentations, improving downstream robustness and efficiency.

Findings

CLeVER improves training efficiency and robustness in downstream tasks.

It achieves state-of-the-art performance on multiple benchmarks.

Equivariant information enhances rotational invariance and model stability.

Abstract

Invariant Contrastive Learning (ICL) methods have achieved impressive performance across various domains. However, the absence of latent space representation for distortion (augmentation)-related information in the latent space makes ICL sub-optimal regarding training efficiency and robustness in downstream tasks. Recent studies suggest that introducing equivariance into Contrastive Learning (CL) can improve overall performance. In this paper, we revisit the roles of augmentation strategies and equivariance in improving CL's efficacy. We propose CLeVER (Contrastive Learning Via Equivariant Representation), a novel equivariant contrastive learning framework compatible with augmentation strategies of arbitrary complexity for various mainstream CL backbone models. Experimental results demonstrate that CLeVER effectively extracts and incorporates equivariant information from practical natural images, thereby improving the training efficiency and robustness of baseline models in downstream tasks and achieving state-of-the-art (SOTA) performance. Moreover, we find that leveraging equivariant information extracted by CLeVER simultaneously enhances rotational invariance and sensitivity across experimental tasks, and helps stabilize the framework when handling complex augmentations, particularly for models with small-scale backbones.