Formalizing the Generalization-Forgetting Trade-off in Continual Learning

TL;DR

This paper models the continual learning trade-off between generalization and forgetting as a dynamic game, providing theoretical insights and proposing a balanced approach that improves upon existing methods.

Contribution

It introduces a formal game-theoretic framework for the trade-off in continual learning and proposes a new balanced method that achieves stable equilibrium.

Findings

Existence of a stable balance point between generalization and forgetting.

The proposed Balanced Continual Learning (BCL) method performs comparably or better than state-of-the-art.

Theoretical proof of stability of the balance point.

Abstract

We formulate the continual learning (CL) problem via dynamic programming and model the trade-off between catastrophic forgetting and generalization as a two-player sequential game. In this approach, player 1 maximizes the cost due to lack of generalization whereas player 2 minimizes the cost due to catastrophic forgetting. We show theoretically that a balance point between the two players exists for each task and that this point is stable (once the balance is achieved, the two players stay at the balance point). Next, we introduce balanced continual learning (BCL), which is designed to attain balance between generalization and forgetting and empirically demonstrate that BCL is comparable to or better than the state of the art.