A Survey of Deep Meta-Learning

TL;DR

This survey provides a comprehensive overview of deep meta-learning techniques, categorizing methods and discussing challenges like evaluation on diverse benchmarks and reducing computational costs.

Contribution

It offers a unified, in-depth overview of current deep meta-learning methods, including theoretical foundations and categorization into metric, model, and optimization-based techniques.

Findings

Categorizes deep meta-learning methods into three main types.

Identifies key open challenges in the field.

Highlights the need for diverse benchmarks and efficiency improvements.

Abstract

Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into i)~metric-, ii)~model-, and iii)~optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.