Self-Attention Message Passing for Contrastive Few-Shot Learning

TL;DR

This paper introduces SAMPTransfer, a novel unsupervised few-shot learning framework using self-attention message passing and optimal transport fine-tuning, achieving state-of-the-art results on miniImagenet and tieredImagenet benchmarks.

Contribution

The paper proposes a new self-attention message passing contrastive learning method combined with optimal transport fine-tuning for unsupervised few-shot learning.

Findings

Sets new state-of-the-art on miniImagenet and tieredImagenet benchmarks.

Achieves up to 7%+ and 5%+ improvements over previous methods.

Outperforms existing unsupervised baselines in cross-domain scenarios.

Abstract

Humans have a unique ability to learn new representations from just a handful of examples with little to no supervision. Deep learning models, however, require an abundance of data and supervision to perform at a satisfactory level. Unsupervised few-shot learning (U-FSL) is the pursuit of bridging this gap between machines and humans. Inspired by the capacity of graph neural networks (GNNs) in discovering complex inter-sample relationships, we propose a novel self-attention based message passing contrastive learning approach (coined as SAMP-CLR) for U-FSL pre-training. We also propose an optimal transport (OT) based fine-tuning strategy (we call OpT-Tune) to efficiently induce task awareness into our novel end-to-end unsupervised few-shot classification framework (SAMPTransfer). Our extensive experimental results corroborate the efficacy of SAMPTransfer in a variety of downstream few-shot classification scenarios, setting a new state-of-the-art for U-FSL on both miniImagenet and tieredImagenet benchmarks, offering up to 7%+ and 5%+ improvements, respectively. Our further investigations also confirm that SAMPTransfer remains on-par with some supervised baselines on miniImagenet and outperforms all existing U-FSL baselines in a challenging cross-domain scenario. Our code can be found in our GitHub repository at https://github.com/ojss/SAMPTransfer/.