Error Sensitivity Modulation based Experience Replay: Mitigating Abrupt Representation Drift in Continual Learning

TL;DR

This paper introduces ESMER, a novel continual learning method that modulates error sensitivity and uses error history to reduce catastrophic forgetting and representation drift, especially under noisy data conditions.

Contribution

It proposes a dual-memory system with error sensitivity modulation and an error-aware reservoir sampling strategy for improved continual learning.

Findings

Reduces forgetting and representation drift at task boundaries.

Enables learning under high label noise conditions.

Maintains better knowledge consolidation across tasks.

Abstract

Humans excel at lifelong learning, as the brain has evolved to be robust to distribution shifts and noise in our ever-changing environment. Deep neural networks (DNNs), however, exhibit catastrophic forgetting and the learned representations drift drastically as they encounter a new task. This alludes to a different error-based learning mechanism in the brain. Unlike DNNs, where learning scales linearly with the magnitude of the error, the sensitivity to errors in the brain decreases as a function of their magnitude. To this end, we propose \textit{ESMER} which employs a principled mechanism to modulate error sensitivity in a dual-memory rehearsal-based system. Concretely, it maintains a memory of past errors and uses it to modify the learning dynamics so that the model learns more from small consistent errors compared to large sudden errors. We also propose \textit{Error-Sensitive Reservoir Sampling} to maintain episodic memory, which leverages the error history to pre-select low-loss samples as candidates for the buffer, which are better suited for retaining information. Empirical results show that ESMER effectively reduces forgetting and abrupt drift in representations at the task boundary by gradually adapting to the new task while consolidating knowledge. Remarkably, it also enables the model to learn under high levels of label noise, which is ubiquitous in real-world data streams.