Meta-Learning for Natural Language Understanding under Continual Learning Framework

TL;DR

This paper explores meta-learning approaches, specifically MAML and OML, within a continual learning framework to improve natural language understanding across multiple tasks, validated on SuperGLUE and GLUE benchmarks.

Contribution

It introduces the application of MAML and OML in a continual learning setting for NLU, which is a novel combination for this domain.

Findings

Meta-learning methods enhance NLU performance.

Validation on SuperGLUE and GLUE benchmarks shows promising results.

Continual learning framework improves task generalization.

Abstract

Neural network has been recognized with its accomplishments on tackling various natural language understanding (NLU) tasks. Methods have been developed to train a robust model to handle multiple tasks to gain a general representation of text. In this paper, we implement the model-agnostic meta-learning (MAML) and Online aware Meta-learning (OML) meta-objective under the continual framework for NLU tasks. We validate our methods on selected SuperGLUE and GLUE benchmark.