NeurIPS'22 Cross-Domain MetaDL competition: Design and baseline results

TL;DR

This paper introduces a new cross-domain meta-learning challenge at NeurIPS'22, with a comprehensive dataset and baseline results, aiming to advance few-shot learning across diverse real-world domains.

Contribution

It presents the design of a cross-domain meta-learning competition, including the Meta-Album dataset with 40 datasets from 10 domains, and baseline results for future research.

Findings

Baseline results established for cross-domain meta-learning.

Meta-Album dataset enables diverse task creation.

Open-sourcing code will facilitate further advancements.

Abstract

We present the design and baseline results for a new challenge in the ChaLearn meta-learning series, accepted at NeurIPS'22, focusing on "cross-domain" meta-learning. Meta-learning aims to leverage experience gained from previous tasks to solve new tasks efficiently (i.e., with better performance, little training data, and/or modest computational resources). While previous challenges in the series focused on within-domain few-shot learning problems, with the aim of learning efficiently N-way k-shot tasks (i.e., N class classification problems with k training examples), this competition challenges the participants to solve "any-way" and "any-shot" problems drawn from various domains (healthcare, ecology, biology, manufacturing, and others), chosen for their humanitarian and societal impact. To that end, we created Meta-Album, a meta-dataset of 40 image classification datasets from 10 domains, from which we carve out tasks with any number of "ways" (within the range 2-20) and any number of "shots" (within the range 1-20). The competition is with code submission, fully blind-tested on the CodaLab challenge platform. The code of the winners will be open-sourced, enabling the deployment of automated machine learning solutions for few-shot image classification across several domains.