Learning continually with representational drift

TL;DR

This paper explores how representational drift in biological neural networks, which involves gradual changes in neural responses over time, can inform the development of continual learning methods in artificial neural networks, highlighting the potential benefits of instability.

Contribution

It proposes that representational drift, a phenomenon observed in biological brains, could be a useful property for artificial systems to balance stability and plasticity in continual learning.

Findings

Representational drift occurs across many species and brain regions.

Drift results from homeostatic turnover and learning-related plasticity.

In artificial networks, drift is compatible with approaches that allow parameter changes.

Abstract

Deep artificial neural networks famously struggle to learn from non-stationary streams of data. Without dedicated mitigation strategies, continual learning is associated with continuous forgetting of previous tasks and a progressive loss of plasticity. Current approaches to continual learning have either focused on increasing the stability of representations of past tasks, or on promoting plasticity for future learning. Paradoxically, while animals including humans achieve a desirable stability-plasticity trade-off, the responses of biological neurons to external stimuli that are associated with stable behaviors gradually change over time. This suggests that, although unstable representations have historically been seen as undesirable in artificial systems, they could be a core property of biological neural networks learning continually. Here, we examine how linking representational drift to continual learning in biological neural networks could inform artificial systems. We highlight the existence of representational drift across numerous animal species and brain regions and propose that drift reflects a mixture of homeostatic turnover and learning-related synaptic plasticity. In particular, we evaluate how plasticity induced by learning new tasks could induce drift in the representation of previous tasks, and how such drift could accumulate across brain regions. In deep artificial neural networks, we propose that representational drift is only compatible with approaches that do not explicitly prevent parameter changes to mitigate forgetting. Remarkably, jointly promoting plasticity while mitigating forgetting could in principle induce representational drift in continual learning. While we argue that drift is a byproduct rather than a solution to incremental learning, its investigation could inform approaches to continual learning in artificial systems.