Deep Meta-Learning: Learning to Learn in the Concept Space

TL;DR

This paper introduces deep meta-learning, which enhances few-shot learning by learning in a concept space using a joint framework of a concept generator, meta-learner, and discriminator, significantly improving performance on image recognition tasks.

Contribution

It proposes a novel deep meta-learning framework that learns high-level concepts for improved few-shot learning, integrating deep representation learning with meta-learning.

Findings

Significant accuracy improvements on CIFAR-100 and CUB-200 datasets.

Enhanced performance of existing meta-learning algorithms in the concept space.

Demonstrated effectiveness across various few-shot image recognition tasks.

Abstract

Few-shot learning remains challenging for meta-learning that learns a learning algorithm (meta-learner) from many related tasks. In this work, we argue that this is due to the lack of a good representation for meta-learning, and propose deep meta-learning to integrate the representation power of deep learning into meta-learning. The framework is composed of three modules, a concept generator, a meta-learner, and a concept discriminator, which are learned jointly. The concept generator, e.g. a deep residual net, extracts a representation for each instance that captures its high-level concept, on which the meta-learner performs few-shot learning, and the concept discriminator recognizes the concepts. By learning to learn in the concept space rather than in the complicated instance space, deep meta-learning can substantially improve vanilla meta-learning, which is demonstrated on various few-shot image recognition problems. For example, on 5-way-1-shot image recognition on CIFAR-100 and CUB-200, it improves Matching Nets from 50.53% and 56.53% to 58.18% and 63.47%, improves MAML from 49.28% and 50.45% to 56.65% and 64.63%, and improves Meta-SGD from 53.83% and 53.34% to 61.62% and 66.95%, respectively.