Representation Learning via Consistent Assignment of Views to Clusters

TL;DR

CARL is an unsupervised visual representation learning method that combines contrastive learning with deep clustering, using online prototype learning to ensure consistent view assignments, leading to superior performance on various benchmarks.

Contribution

CARL introduces an online prototype learning approach that unifies contrastive learning and deep clustering without non-differentiable algorithms.

Findings

Outperforms competitors in linear evaluation benchmarks.

Effective in semi-supervised learning scenarios.

Excels in transfer learning tasks.

Abstract

We introduce Consistent Assignment for Representation Learning (CARL), an unsupervised learning method to learn visual representations by combining ideas from self-supervised contrastive learning and deep clustering. By viewing contrastive learning from a clustering perspective, CARL learns unsupervised representations by learning a set of general prototypes that serve as energy anchors to enforce different views of a given image to be assigned to the same prototype. Unlike contemporary work on contrastive learning with deep clustering, CARL proposes to learn the set of general prototypes in an online fashion, using gradient descent without the necessity of using non-differentiable algorithms or K-Means to solve the cluster assignment problem. CARL surpasses its competitors in many representations learning benchmarks, including linear evaluation, semi-supervised learning, and transfer learning.