Adversarial Pseudo-replay for Exemplar-free Class-incremental Learning

TL;DR

This paper introduces adversarial pseudo-replay, a novel method for exemplar-free class-incremental learning that synthesizes pseudo-replay images via adversarial attacks to mitigate catastrophic forgetting without storing previous data.

Contribution

The paper proposes adversarial pseudo-replay (APR), a new online pseudo-replay technique using adversarial attacks and covariance calibration to improve EFCIL performance.

Findings

Achieves state-of-the-art results on standard EFCIL benchmarks.

Effectively balances stability and plasticity without storing old data.

Outperforms existing methods in cold-start scenarios.

Abstract

Exemplar-free class-incremental learning (EFCIL) aims to retain old knowledge acquired in the previous task while learning new classes, without storing the previous images due to storage constraints or privacy concerns. In EFCIL, the plasticity-stability dilemma, learning new tasks versus catastrophic forgetting, is a significant challenge, primarily due to the unavailability of images from earlier tasks. In this paper, we introduce adversarial pseudo-replay (APR), a method that perturbs the images of the new task with adversarial attack, to synthesize the pseudo-replay images online without storing any replay samples. During the new task training, the adversarial attack is conducted on the new task images with augmented old class mean prototypes as targets, and the resulting images are used for knowledge distillation to prevent semantic drift. Moreover, we calibrate the covariance matrices to compensate for the semantic drift after each task, by learning a transfer matrix on the pseudo-replay samples. Our method reconciles stability and plasticity, achieving state-of-the-art on challenging cold-start settings of the standard EFCIL benchmarks.