Few-Max: Few-Shot Domain Adaptation for Unsupervised Contrastive Representation Learning

TL;DR

Few-Max introduces a domain adaptation technique for contrastive self-supervised learning that effectively improves representation quality in few-shot target domain scenarios, outperforming existing methods across multiple datasets.

Contribution

The paper proposes Few-Max, a novel domain adaptation method specifically designed for self-supervised contrastive learning in few-shot settings, addressing overfitting and generalization issues.

Findings

Few-Max outperforms other methods on ImageNet, VisDA, and fastMRI datasets.

It effectively adapts representations to target domains with limited data.

The approach demonstrates consistent improvements in representation quality.

Abstract

Contrastive self-supervised learning methods learn to map data points such as images into non-parametric representation space without requiring labels. While highly successful, current methods require a large amount of data in the training phase. In situations where the target training set is limited in size, generalization is known to be poor. Pretraining on a large source data set and fine-tuning on the target samples is prone to overfitting in the few-shot regime, where only a small number of target samples are available. Motivated by this, we propose a domain adaption method for self-supervised contrastive learning, termed Few-Max, to address the issue of adaptation to a target distribution under few-shot learning. To quantify the representation quality, we evaluate Few-Max on a range of source and target datasets, including ImageNet, VisDA, and fastMRI, on which Few-Max consistently outperforms other approaches.