Learning Continually by Spectral Regularization

TL;DR

This paper introduces a spectral regularizer that maintains neural network trainability during continual learning by controlling singular values, leading to improved performance and robustness across tasks.

Contribution

A novel spectral regularizer inspired by singular value properties at initialization, enhancing continual learning by preserving trainability and performance.

Findings

Spectral regularization sustains trainability across tasks.

It maintains gradient diversity during training.

It improves generalization in continual learning settings.

Abstract

Loss of plasticity is a phenomenon where neural networks can become more difficult to train over the course of learning. Continual learning algorithms seek to mitigate this effect by sustaining good performance while maintaining network trainability. We develop a new technique for improving continual learning inspired by the observation that the singular values of the neural network parameters at initialization are an important factor for trainability during early phases of learning. From this perspective, we derive a new spectral regularizer for continual learning that better sustains these beneficial initialization properties throughout training. In particular, the regularizer keeps the maximum singular value of each layer close to one. Spectral regularization directly ensures that gradient diversity is maintained throughout training, which promotes continual trainability, while minimally interfering with performance in a single task. We present an experimental analysis that shows how the proposed spectral regularizer can sustain trainability and performance across a range of model architectures in continual supervised and reinforcement learning settings. Spectral regularization is less sensitive to hyperparameters while demonstrating better training in individual tasks, sustaining trainability as new tasks arrive, and achieving better generalization performance.