Principal Gradient Direction and Confidence Reservoir Sampling for Continual Learning

TL;DR

This paper introduces Principal Gradient Direction and Confidence Reservoir Sampling to enhance Experience Replay in task-free online continual learning, significantly reducing forgetting and improving accuracy on multiple datasets.

Contribution

It proposes a unified proximal gradient framework and two novel techniques, PGD and CRS, to improve replay-based continual learning methods.

Findings

Increases average accuracy by up to 7.9%.

Reduces forgetting by up to 15.4%.

Consistently outperforms state-of-the-art methods.

Abstract

Task-free online continual learning aims to alleviate catastrophic forgetting of the learner on a non-iid data stream. Experience Replay (ER) is a SOTA continual learning method, which is broadly used as the backbone algorithm for other replay-based methods. However, the training strategy of ER is too simple to take full advantage of replayed examples and its reservoir sampling strategy is also suboptimal. In this work, we propose a general proximal gradient framework so that ER can be viewed as a special case. We further propose two improvements accordingly: Principal Gradient Direction (PGD) and Confidence Reservoir Sampling (CRS). In Principal Gradient Direction, we optimize a target gradient that not only represents the major contribution of past gradients, but also retains the new knowledge of the current gradient. We then present Confidence Reservoir Sampling for maintaining a more informative memory buffer based on a margin-based metric that measures the value of stored examples. Experiments substantiate the effectiveness of both our improvements and our new algorithm consistently boosts the performance of MIR-replay, a SOTA ER-based method: our algorithm increases the average accuracy up to 7.9% and reduces forgetting up to 15.4% on four datasets.