Autoencoder-Based Hybrid Replay for Class-Incremental Learning

TL;DR

This paper introduces an autoencoder-based hybrid replay strategy for class-incremental learning that reduces memory requirements and maintains state-of-the-art performance by compressing exemplar data in latent space.

Contribution

The proposed hybrid autoencoder (HAE) enables efficient data compression and dual modeling for classification and replay, improving memory efficiency in class-incremental learning.

Findings

Outperforms recent baselines across multiple benchmarks.

Achieves $ ext{O}(0.1 t)$ memory complexity, reducing resource usage.

Maintains state-of-the-art accuracy with lower memory and compute costs.

Abstract

In class-incremental learning (CIL), effective incremental learning strategies are essential to mitigate task confusion and catastrophic forgetting, especially as the number of tasks $t$ increases. Current exemplar replay strategies impose $\mathcal{O}(t)$ memory/compute complexities. We propose an autoencoder-based hybrid replay (AHR) strategy that leverages our new hybrid autoencoder (HAE) to function as a compressor to alleviate the requirement for large memory, achieving $\mathcal{O}(0.1 t)$ at the worst case with the computing complexity of $\mathcal{O}(t)$ while accomplishing state-of-the-art performance. The decoder later recovers the exemplar data stored in the latent space, rather than in raw format. Additionally, HAE is designed for both discriminative and generative modeling, enabling classification and replay capabilities, respectively. HAE adopts the charged particle system energy minimization equations and repulsive force algorithm for the incremental embedding and distribution of new class centroids in its latent space. Our results demonstrate that AHR consistently outperforms recent baselines across multiple benchmarks while operating with the same memory/compute budgets. The source code is included in the supplementary material and will be open-sourced upon publication.