The Role of Global Labels in Few-Shot Classification and How to Infer Them

TL;DR

This paper explores the importance of global labels in few-shot classification, providing theoretical insights and introducing Meta Label Learning (MeLa), a framework that infers global labels to improve model robustness.

Contribution

The paper offers a theoretical understanding of pre-training benefits in meta-learning and proposes MeLa, a novel method for inferring global labels in few-shot tasks.

Findings

MeLa performs competitively with existing methods.

Global label inference enhances few-shot learning robustness.

Extensive ablations validate key properties of MeLa.

Abstract

Few-shot learning is a central problem in meta-learning, where learners must quickly adapt to new tasks given limited training data. Recently, feature pre-training has become a ubiquitous component in state-of-the-art meta-learning methods and is shown to provide significant performance improvement. However, there is limited theoretical understanding of the connection between pre-training and meta-learning. Further, pre-training requires global labels shared across tasks, which may be unavailable in practice. In this paper, we show why exploiting pre-training is theoretically advantageous for meta-learning, and in particular the critical role of global labels. This motivates us to propose Meta Label Learning (MeLa), a novel meta-learning framework that automatically infers global labels to obtains robust few-shot models. Empirically, we demonstrate that MeLa is competitive with existing methods and provide extensive ablation experiments to highlight its key properties.