MxML: Mixture of Meta-Learners for Few-Shot Classification

TL;DR

This paper introduces MxML, a mixture of meta-learners with a weight prediction network that enhances few-shot classification, especially for out-of-distribution tasks, outperforming existing methods.

Contribution

The paper proposes a novel ensemble approach, MxML, that dynamically combines meta-learners trained on different task distributions to improve few-shot learning performance.

Findings

MxML outperforms state-of-the-art meta-learners on various datasets.

MxML effectively handles out-of-distribution tasks.

Ensemble of meta-learners enhances robustness in few-shot classification.

Abstract

A meta-model is trained on a distribution of similar tasks such that it learns an algorithm that can quickly adapt to a novel task with only a handful of labeled examples. Most of current meta-learning methods assume that the meta-training set consists of relevant tasks sampled from a single distribution. In practice, however, a new task is often out of the task distribution, yielding a performance degradation. One way to tackle this problem is to construct an ensemble of meta-learners such that each meta-learner is trained on different task distribution. In this paper we present a method for constructing a mixture of meta-learners (MxML), where mixing parameters are determined by the weight prediction network (WPN) optimized to improve the few-shot classification performance. Experiments on various datasets demonstrate that MxML significantly outperforms state-of-the-art meta-learners, or their naive ensemble in the case of out-of-distribution as well as in-distribution tasks.