AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning

TL;DR

AdaER is a novel adaptive experience replay method for continual lifelong learning that effectively mitigates catastrophic forgetting by selectively replaying conflicting memories and maximizing information entropy in memory buffers.

Contribution

The paper introduces AdaER, combining contextually-cued memory recall and entropy-balanced reservoir sampling to improve continual learning performance.

Findings

AdaER outperforms existing baselines in class-incremental learning tasks.

It effectively reduces catastrophic forgetting.

Demonstrates superior retention of previous knowledge.

Abstract

Continual lifelong learning is an machine learning framework inspired by human learning, where learners are trained to continuously acquire new knowledge in a sequential manner. However, the non-stationary nature of streaming training data poses a significant challenge known as catastrophic forgetting, which refers to the rapid forgetting of previously learned knowledge when new tasks are introduced. While some approaches, such as experience replay (ER), have been proposed to mitigate this issue, their performance remains limited, particularly in the class-incremental scenario which is considered natural and highly challenging. In this paper, we present a novel algorithm, called adaptive-experience replay (AdaER), to address the challenge of continual lifelong learning. AdaER consists of two stages: memory replay and memory update. In the memory replay stage, AdaER introduces a contextually-cued memory recall (C-CMR) strategy, which selectively replays memories that are most conflicting with the current input data in terms of both data and task. Additionally, AdaER incorporates an entropy-balanced reservoir sampling (E-BRS) strategy to enhance the performance of the memory buffer by maximizing information entropy. To evaluate the effectiveness of AdaER, we conduct experiments on established supervised continual lifelong learning benchmarks, specifically focusing on class-incremental learning scenarios. The results demonstrate that AdaER outperforms existing continual lifelong learning baselines, highlighting its efficacy in mitigating catastrophic forgetting and improving learning performance.