The Expanding Scope of the Stability Gap: Unveiling its Presence in Joint Incremental Learning of Homogeneous Tasks

TL;DR

This paper reveals that the stability gap, previously observed in continual learning, also occurs in joint incremental training of homogeneous tasks, highlighting challenges in optimization paths and performance stability.

Contribution

It demonstrates the presence of the stability gap in joint incremental learning and analyzes the optimization dynamics, offering insights into potential solutions.

Findings

Stability gap occurs in joint incremental training of homogeneous tasks.

A low-loss linear path exists but SGD does not follow it.

Batch-wise analysis provides insights into optimization challenges.

Abstract

Recent research identified a temporary performance drop on previously learned tasks when transitioning to a new one. This drop is called the stability gap and has great consequences for continual learning: it complicates the direct employment of continually learning since the worse-case performance at task-boundaries is dramatic, it limits its potential as an energy-efficient training paradigm, and finally, the stability drop could result in a reduced final performance of the algorithm. In this paper, we show that the stability gap also occurs when applying joint incremental training of homogeneous tasks. In this scenario, the learner continues training on the same data distribution and has access to all data from previous tasks. In addition, we show that in this scenario, there exists a low-loss linear path to the next minima, but that SGD optimization does not choose this path. We perform further analysis including a finer batch-wise analysis which could provide insights towards potential solution directions.