Continual Learning with Node-Importance based Adaptive Group Sparse Regularization

TL;DR

This paper introduces AGS-CL, a novel continual learning method that adaptively regularizes nodes based on importance, effectively preventing catastrophic forgetting and outperforming existing methods on various benchmarks.

Contribution

The paper presents a new adaptive group sparsity regularization technique for continual learning that dynamically updates node importance and re-initializes unimportant nodes to improve performance.

Findings

AGS-CL outperforms state-of-the-art baselines on multiple benchmarks.

The method uses less memory for regularization parameters.

Explicit control of model capacity helps prevent catastrophic forgetting.

Abstract

We propose a novel regularization-based continual learning method, dubbed as Adaptive Group Sparsity based Continual Learning (AGS-CL), using two group sparsity-based penalties. Our method selectively employs the two penalties when learning each node based its the importance, which is adaptively updated after learning each new task. By utilizing the proximal gradient descent method for learning, the exact sparsity and freezing of the model is guaranteed, and thus, the learner can explicitly control the model capacity as the learning continues. Furthermore, as a critical detail, we re-initialize the weights associated with unimportant nodes after learning each task in order to prevent the negative transfer that causes the catastrophic forgetting and facilitate efficient learning of new tasks. Throughout the extensive experimental results, we show that our AGS-CL uses much less additional memory space for storing the regularization parameters, and it significantly outperforms several state-of-the-art baselines on representative continual learning benchmarks for both supervised and reinforcement learning tasks.