TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning

TL;DR

TEAL is a novel exemplar selection strategy that significantly improves the performance of experience replay in continual learning, especially with small memory buffers, by prioritizing typical data to mitigate catastrophic forgetting.

Contribution

Introduces TEAL, a new exemplar selection method that enhances small-buffer experience replay in continual learning, outperforming existing strategies and achieving state-of-the-art results.

Findings

TEAL improves accuracy of class-incremental methods with small buffers.

TEAL outperforms other selection strategies in experiments.

State-of-the-art performance with 1-3 exemplars per class.

Abstract

Continual Learning is an unresolved challenge, whose relevance increases when considering modern applications. Unlike the human brain, trained deep neural networks suffer from a phenomenon called catastrophic forgetting, wherein they progressively lose previously acquired knowledge upon learning new tasks. To mitigate this problem, numerous methods have been developed, many relying on the replay of past exemplars during new task training. However, as the memory allocated for replay decreases, the effectiveness of these approaches diminishes. On the other hand, maintaining a large memory for the purpose of replay is inefficient and often impractical. Here we introduce TEAL, a novel approach to populate the memory with exemplars, that can be integrated with various experience-replay methods and significantly enhance their performance with small memory buffers. We show that TEAL enhances the average accuracy of existing class-incremental methods and outperforms other selection strategies, achieving state-of-the-art performance even with small memory buffers of 1-3 exemplars per class in the final task. This confirms our initial hypothesis that when memory is scarce, it is best to prioritize the most typical data. Code is available at this https URL: https://github.com/shahariel/TEAL.