Exploring the Stability Gap in Continual Learning: The Role of the Classification Head

TL;DR

This paper investigates the stability gap in continual learning, revealing that the classification head significantly influences this phenomenon, and introduces the nearest-mean classifier to improve stability and reduce bias.

Contribution

The study introduces the nearest-mean classifier (NMC) as a tool to analyze and improve the stability of neural networks in continual learning, emphasizing the role of the classification head.

Findings

NMC improves final performance across benchmarks.

NMC enhances training stability and reduces task-recency bias.

The classification head is the primary contributor to the stability gap.

Abstract

Continual learning (CL) has emerged as a critical area in machine learning, enabling neural networks to learn from evolving data distributions while mitigating catastrophic forgetting. However, recent research has identified the stability gap -- a phenomenon where models initially lose performance on previously learned tasks before partially recovering during training. Such learning dynamics are contradictory to the intuitive understanding of stability in continual learning where one would expect the performance to degrade gradually instead of rapidly decreasing and then partially recovering later. To better understand and alleviate the stability gap, we investigate it at different levels of the neural network architecture, particularly focusing on the role of the classification head. We introduce the nearest-mean classifier (NMC) as a tool to attribute the influence of the backbone and the classification head on the stability gap. Our experiments demonstrate that NMC not only improves final performance, but also significantly enhances training stability across various continual learning benchmarks, including CIFAR100, ImageNet100, CUB-200, and FGVC Aircrafts. Moreover, we find that NMC also reduces task-recency bias. Our analysis provides new insights into the stability gap and suggests that the primary contributor to this phenomenon is the linear head, rather than the insufficient representation learning.