Self-Normalized Resets for Plasticity in Continual Learning

TL;DR

This paper introduces Self-Normalized Resets (SNR), an adaptive method to mitigate plasticity loss in continual learning by resetting neurons based on firing rate evidence, demonstrating superior performance and robustness across tasks.

Contribution

The paper proposes SNR, a novel reset mechanism for neurons in continual learning, supported by a hypothesis test and theoretical analysis showing its effectiveness and robustness.

Findings

SNR outperforms competing algorithms in various continual learning tasks.

SNR is robust to its hyperparameter, the rejection percentile threshold.

Theoretical analysis shows idealized SNR can learn ReLUs even from adversarial initializations.

Abstract

Plasticity Loss is an increasingly important phenomenon that refers to the empirical observation that as a neural network is continually trained on a sequence of changing tasks, its ability to adapt to a new task diminishes over time. We introduce Self-Normalized Resets (SNR), a simple adaptive algorithm that mitigates plasticity loss by resetting a neuron's weights when evidence suggests its firing rate has effectively dropped to zero. Across a battery of continual learning problems and network architectures, we demonstrate that SNR consistently attains superior performance compared to its competitor algorithms. We also demonstrate that SNR is robust to its sole hyperparameter, its rejection percentile threshold, while competitor algorithms show significant sensitivity. SNR's threshold-based reset mechanism is motivated by a simple hypothesis test that we derive. Seen through the lens of this hypothesis test, competing reset proposals yield suboptimal error rates in correctly detecting inactive neurons, potentially explaining our experimental observations. We also conduct a theoretical investigation of the optimization landscape for the problem of learning a single ReLU. We show that even when initialized adversarially, an idealized version of SNR learns the target ReLU, while regularization based approaches can fail to learn.