Episodic memory for continual model learning

TL;DR

This paper introduces a method using episodic memory buffers to enable online model selection in continual learning, overcoming memory constraints by retaining key data points that facilitate model switching.

Contribution

It proposes a novel approach where episodic memory is used to retain critical data for effective online model selection despite limited memory capacity.

Findings

Episodic memory buffers can improve online model selection.

Optimized data retention enhances model switching accuracy.

Method reduces memory requirements compared to full data storage.

Abstract

Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian inference provides a principled method for model selection and it converges on the same posterior for both off-line (i.e. batch) and online learning. However, maintaining a parameter posterior for each model in parallel has in general an even higher memory cost than storing the entire data set and is consequently clearly unfeasible. Alternatively, maintaining only a limited set of models in memory could limit memory requirements. However, sufficient statistics for one model will usually be insufficient for fitting a different kind of model, meaning that the agent loses information with each model change. We propose that episodic memory can circumvent the challenge of limited memory-capacity online model selection by retaining a selected subset of data points. We design a method to compute the quantities necessary for model selection even when the data is discarded and only statistics of one (or few) learnt models are available. We demonstrate on a simple model that a limited-sized episodic memory buffer, when the content is optimised to retain data with statistics not matching the current representation, can resolve the fundamental challenge of online model selection.