Learning to Continually Learn

TL;DR

This paper introduces ANML, a neuromodulated meta-learning algorithm inspired by brain processes, which enables neural networks to learn sequential tasks without catastrophic forgetting by dynamically gating activations.

Contribution

It proposes a novel meta-learning approach that learns an activation-gating function to prevent forgetting, inspired by neuromodulatory mechanisms in the brain.

Findings

Achieves state-of-the-art continual learning performance

Successfully learns over 600 classes without catastrophic forgetting

Operates effectively at scale with thousands of updates

Abstract

Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it. Much work has gone towards preventing the default tendency of machine learning models to catastrophically forget, yet virtually all such work involves manually-designed solutions to the problem. We instead advocate meta-learning a solution to catastrophic forgetting, allowing AI to learn to continually learn. Inspired by neuromodulatory processes in the brain, we propose A Neuromodulated Meta-Learning Algorithm (ANML). It differentiates through a sequential learning process to meta-learn an activation-gating function that enables context-dependent selective activation within a deep neural network. Specifically, a neuromodulatory (NM) neural network gates the forward pass of another (otherwise normal) neural network called the prediction learning network (PLN). The NM network also thus indirectly controls selective plasticity (i.e. the backward pass of) the PLN. ANML enables continual learning without catastrophic forgetting at scale: it produces state-of-the-art continual learning performance, sequentially learning as many as 600 classes (over 9,000 SGD updates).