Memory Population in Continual Learning via Outlier Elimination

TL;DR

This paper proposes Memory Outlier Elimination (MOE), a novel method to improve continual learning by removing noisy outliers from memory buffers, leading to better performance on standard datasets.

Contribution

The paper introduces MOE, a new outlier removal technique for memory buffers that enhances continual learning by selecting more representative samples.

Findings

MOE outperforms existing memory population methods on CIFAR-10, CIFAR-100, and CORe50.

Removing outliers improves the generalization of continual learning models.

High homogeneity in feature space correlates with better class representation.

Abstract

Catastrophic forgetting, the phenomenon of forgetting previously learned tasks when learning a new one, is a major hurdle in developing continual learning algorithms. A popular method to alleviate forgetting is to use a memory buffer, which stores a subset of previously learned task examples for use during training on new tasks. The de facto method of filling memory is by randomly selecting previous examples. However, this process could introduce outliers or noisy samples that could hurt the generalization of the model. This paper introduces Memory Outlier Elimination (MOE), a method for identifying and eliminating outliers in the memory buffer by choosing samples from label-homogeneous subpopulations. We show that a space with a high homogeneity is related to a feature space that is more representative of the class distribution. In practice, MOE removes a sample if it is surrounded by samples from different labels. We demonstrate the effectiveness of MOE on CIFAR-10, CIFAR-100, and CORe50, outperforming previous well-known memory population methods.