Scalable Adversarial Online Continual Learning

TL;DR

This paper introduces SCALE, a scalable adversarial online continual learning method that uses a parameter generator and a single discriminator, enabling efficient online learning with improved accuracy and reduced complexity.

Contribution

SCALE offers a novel, scalable approach to adversarial continual learning by replacing task-specific networks with a parameter generator and a single discriminator, suitable for online settings.

Findings

SCALE outperforms existing methods in accuracy.

SCALE reduces training complexity and execution time.

SCALE effectively handles online continual learning scenarios.

Abstract

Adversarial continual learning is effective for continual learning problems because of the presence of feature alignment process generating task-invariant features having low susceptibility to the catastrophic forgetting problem. Nevertheless, the ACL method imposes considerable complexities because it relies on task-specific networks and discriminators. It also goes through an iterative training process which does not fit for online (one-epoch) continual learning problems. This paper proposes a scalable adversarial continual learning (SCALE) method putting forward a parameter generator transforming common features into task-specific features and a single discriminator in the adversarial game to induce common features. The training process is carried out in meta-learning fashions using a new combination of three loss functions. SCALE outperforms prominent baselines with noticeable margins in both accuracy and execution time.