Supervised Momentum Contrastive Learning for Few-Shot Classification

TL;DR

This paper introduces SUPMOCO, a novel framework combining contrastive and supervised learning to improve few-shot classification, achieving state-of-the-art results on META-DATASET by leveraging multiple data sources.

Contribution

The paper proposes SUPMOCO, a simple yet scalable method that effectively combines contrastive and supervised learning for few-shot classification, enhancing generalization across domains.

Findings

Achieves state-of-the-art on META-DATASET

Combines data from multiple domains effectively

Scales better than previous methods

Abstract

Few-shot learning aims to transfer information from one task to enable generalization on novel tasks given a few examples. This information is present both in the domain and the class labels. In this work we investigate the complementary roles of these two sources of information by combining instance-discriminative contrastive learning and supervised learning in a single framework called Supervised Momentum Contrastive learning (SUPMOCO). Our approach avoids a problem observed in supervised learning where information in images not relevant to the task is discarded, which hampers their generalization to novel tasks. We show that (self-supervised) contrastive learning and supervised learning are mutually beneficial, leading to a new state-of-the-art on the META-DATASET - a recently introduced benchmark for few-shot learning. Our method is based on a simple modification of MOCO and scales better than prior work on combining supervised and self-supervised learning. This allows us to easily combine data from multiple domains leading to further improvements.