Online Coreset Selection for Rehearsal-based Continual Learning

TL;DR

This paper introduces Online Coreset Selection (OCS), an effective method for selecting representative data samples in rehearsal-based continual learning to improve task adaptation and reduce catastrophic forgetting, especially in challenging data scenarios.

Contribution

The paper proposes a novel online coreset selection method that dynamically chooses informative samples to enhance continual learning performance and mitigate forgetting.

Findings

OCS outperforms strong baselines on standard datasets.

OCS effectively handles imbalanced and noisy data scenarios.

Sample efficiency is improved with the proposed method.

Abstract

A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection problem becomes even more important under realistic settings, such as imbalanced continual learning or noisy data scenarios. To tackle this problem, we propose Online Coreset Selection (OCS), a simple yet effective method that selects the most representative and informative coreset at each iteration and trains them in an online manner. Our proposed method maximizes the model's adaptation to a current dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.