# Sequential mastery of multiple visual tasks: Networks naturally learn to learn and forget to forget

## TL;DR

This paper investigates how convolutional neural networks learn multiple visual tasks sequentially, revealing that they become better at learning new tasks and forget less of previous ones as they gain more experience, mimicking human learning.

## Contribution

It demonstrates that neural networks naturally improve in learning efficiency and reduce forgetting with increased exposure to related tasks, highlighting scalable continual learning behaviors.

## Key findings

- Forward facilitation increases with number of learned tasks.
- Backward interference diminishes as more tasks are learned.
- Broader domain exposure enhances both learning and retention.

## Abstract

We explore the behavior of a standard convolutional neural net in a continual-learning setting that introduces visual classification tasks sequentially and requires the net to master new tasks while preserving mastery of previously learned tasks. This setting corresponds to that which human learners face as they acquire domain expertise serially, for example, as an individual studies a textbook. Through simulations involving sequences of ten related visual tasks, we find reason for optimism that nets will scale well as they advance from having a single skill to becoming multi-skill domain experts. We observe two key phenomena. First, \emph{forward facilitation}---the accelerated learning of task $n+1$ having learned $n$ previous tasks---grows with $n$. Second, \emph{backward interference}---the forgetting of the $n$ previous tasks when learning task $n+1$---diminishes with $n$. Amplifying forward facilitation is the goal of research on metalearning, and attenuating backward interference is the goal of research on catastrophic forgetting. We find that both of these goals are attained simply through broader exposure to a domain.