Meta-learning for Few-shot Natural Language Processing: A Survey

TL;DR

This survey reviews meta-learning techniques tailored for few-shot NLP tasks, highlighting recent progress, key datasets, and defining the core concepts to advance research in low-resource language understanding.

Contribution

It provides a comprehensive overview of meta-learning methods specifically for few-shot NLP, clarifies definitions, and summarizes progress and datasets in this domain.

Findings

Summarizes recent meta-learning approaches for few-shot NLP

Provides a clear taxonomy and definitions for the field

Lists key datasets used in few-shot NLP research

Abstract

Few-shot natural language processing (NLP) refers to NLP tasks that are accompanied with merely a handful of labeled examples. This is a real-world challenge that an AI system must learn to handle. Usually we rely on collecting more auxiliary information or developing a more efficient learning algorithm. However, the general gradient-based optimization in high capacity models, if training from scratch, requires many parameter-updating steps over a large number of labeled examples to perform well (Snell et al., 2017). If the target task itself cannot provide more information, how about collecting more tasks equipped with rich annotations to help the model learning? The goal of meta-learning is to train a model on a variety of tasks with rich annotations, such that it can solve a new task using only a few labeled samples. The key idea is to train the model's initial parameters such that the model has maximal performance on a new task after the parameters have been updated through zero or a couple of gradient steps. There are already some surveys for meta-learning, such as (Vilalta and Drissi, 2002; Vanschoren, 2018; Hospedales et al., 2020). Nevertheless, this paper focuses on NLP domain, especially few-shot applications. We try to provide clearer definitions, progress summary and some common datasets of applying meta-learning to few-shot NLP.