GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning

TL;DR

This paper introduces Gradient Coreset Replay (GCR), a novel method for replay buffer selection in continual learning that significantly reduces catastrophic forgetting by maintaining a gradient-approximating subset, achieving notable accuracy improvements.

Contribution

GCR presents a new gradient-based coreset selection strategy for replay buffers, improving continual learning performance over existing methods.

Findings

Achieves 2-4% accuracy gains in offline continual learning.

Attains up to 5% improvements in online/streaming settings.

Enhances performance with supervised contrastive loss.

Abstract

Continual learning (CL) aims to develop techniques by which a single model adapts to an increasing number of tasks encountered sequentially, thereby potentially leveraging learnings across tasks in a resource-efficient manner. A major challenge for CL systems is catastrophic forgetting, where earlier tasks are forgotten while learning a new task. To address this, replay-based CL approaches maintain and repeatedly retrain on a small buffer of data selected across encountered tasks. We propose Gradient Coreset Replay (GCR), a novel strategy for replay buffer selection and update using a carefully designed optimization criterion. Specifically, we select and maintain a "coreset" that closely approximates the gradient of all the data seen so far with respect to current model parameters, and discuss key strategies needed for its effective application to the continual learning setting. We show significant gains (2%-4% absolute) over the state-of-the-art in the well-studied offline continual learning setting. Our findings also effectively transfer to online / streaming CL settings, showing upto 5% gains over existing approaches. Finally, we demonstrate the value of supervised contrastive loss for continual learning, which yields a cumulative gain of up to 5% accuracy when combined with our subset selection strategy.