Overcoming the Stability Gap in Continual Learning

TL;DR

This paper investigates the stability gap in continual learning for large pre-trained neural networks, proposing a method to reduce this gap and improve computational efficiency in real-world applications.

Contribution

The paper identifies the stability gap as a key obstacle in continual learning and introduces a novel method that significantly reduces this gap in large-scale experiments.

Findings

The proposed method reduces the stability gap in class incremental learning.

It greatly improves computational efficiency in large-scale continual learning.

The approach aligns continual learning with practical production needs.

Abstract

Pre-trained deep neural networks (DNNs) are being widely deployed by industry for making business decisions and to serve users; however, a major problem is model decay, where the DNN's predictions become more erroneous over time, resulting in revenue loss or unhappy users. To mitigate model decay, DNNs are retrained from scratch using old and new data. This is computationally expensive, so retraining happens only once performance significantly decreases. Here, we study how continual learning (CL) could potentially overcome model decay in large pre-trained DNNs and greatly reduce computational costs for keeping DNNs up-to-date. We identify the "stability gap" as a major obstacle in our setting. The stability gap refers to a phenomenon where learning new data causes large drops in performance for past tasks before CL mitigation methods eventually compensate for this drop. We test two hypotheses to investigate the factors influencing the stability gap and identify a method that vastly reduces this gap. In large-scale experiments for both easy and hard CL distributions (e.g., class incremental learning), we demonstrate that our method reduces the stability gap and greatly increases computational efficiency. Our work aligns CL with the goals of the production setting, where CL is needed for many applications.