Multi-Level Contrastive Learning for Few-Shot Problems

TL;DR

This paper introduces a multi-level contrastive learning method that applies contrastive losses at various encoder layers to generate multiple representations, significantly improving few-shot learning performance on mini-ImageNet and tiered-ImageNet datasets.

Contribution

It presents a novel multi-level contrastive learning approach that leverages multiple encoder layers and ensemble methods for enhanced few-shot learning.

Findings

Achieved state-of-the-art results on mini-ImageNet.

Outperformed previous single-level contrastive learning methods.

Demonstrated the effectiveness of multi-level representations in few-shot tasks.

Abstract

Contrastive learning is a discriminative approach that aims at grouping similar samples closer and diverse samples far from each other. It it an efficient technique to train an encoder generating distinguishable and informative representations, and it may even increase the encoder's transferability. Most current applications of contrastive learning benefit only a single representation from the last layer of an encoder.In this paper, we propose a multi-level contrasitive learning approach which applies contrastive losses at different layers of an encoder to learn multiple representations from the encoder. Afterward, an ensemble can be constructed to take advantage of the multiple representations for the downstream tasks. We evaluated the proposed method on few-shot learning problems and conducted experiments using the mini-ImageNet and the tiered-ImageNet datasets. Our model achieved the new state-of-the-art results for both datasets, comparing to previous regular, ensemble, and contrastive learing (single-level) based approaches.