Generalizing from a Few Examples: A Survey on Few-Shot Learning

TL;DR

This survey comprehensively reviews Few-Shot Learning (FSL), highlighting its core challenge of unreliable empirical risk in small data scenarios and categorizing methods based on how prior knowledge is utilized.

Contribution

It provides a formal definition of FSL, a taxonomy of methods, and discusses future directions, offering a thorough understanding of the field and its challenges.

Findings

FSL methods are categorized into data, model, and algorithm approaches.

Prior knowledge helps address the unreliability of empirical risk in FSL.

The survey identifies promising future research directions in FSL.

Abstract

Machine learning has been highly successful in data-intensive applications but is often hampered when the data set is small. Recently, Few-Shot Learning (FSL) is proposed to tackle this problem. Using prior knowledge, FSL can rapidly generalize to new tasks containing only a few samples with supervised information. In this paper, we conduct a thorough survey to fully understand FSL. Starting from a formal definition of FSL, we distinguish FSL from several relevant machine learning problems. We then point out that the core issue in FSL is that the empirical risk minimized is unreliable. Based on how prior knowledge can be used to handle this core issue, we categorize FSL methods from three perspectives: (i) data, which uses prior knowledge to augment the supervised experience; (ii) model, which uses prior knowledge to reduce the size of the hypothesis space; and (iii) algorithm, which uses prior knowledge to alter the search for the best hypothesis in the given hypothesis space. With this taxonomy, we review and discuss the pros and cons of each category. Promising directions, in the aspects of the FSL problem setups, techniques, applications and theories, are also proposed to provide insights for future research.