On Tiny Episodic Memories in Continual Learning

TL;DR

This paper demonstrates that in continual learning, even tiny episodic memories combined with current task data significantly improve performance, outperforming complex methods and enhancing generalization.

Contribution

It empirically shows that a simple joint training approach with minimal episodic memory surpasses specialized CL methods across multiple benchmarks.

Findings

Tiny episodic memories improve continual learning performance.

Joint training on current data and memory outperforms complex CL methods.

Repetitive training on small memories enhances generalization.

Abstract

In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learning algorithms. But for a successful knowledge transfer, the learner needs to remember how to perform previous tasks. One way to endow the learner the ability to perform tasks seen in the past is to store a small memory, dubbed episodic memory, that stores few examples from previous tasks and then to replay these examples when training for future tasks. In this work, we empirically analyze the effectiveness of a very small episodic memory in a CL setup where each training example is only seen once. Surprisingly, across four rather different supervised learning benchmarks adapted to CL, a very simple baseline, that jointly trains on both examples from the current task as well as examples stored in the episodic memory, significantly outperforms specifically designed CL approaches with and without episodic memory. Interestingly, we find that repetitive training on even tiny memories of past tasks does not harm generalization, on the contrary, it improves it, with gains between 7\% and 17\% when the memory is populated with a single example per class.