Information-Theoretic Generalization Bounds of Replay-based Continual Learning

TL;DR

This paper develops an information-theoretic framework to analyze and bound the generalization error in replay-based continual learning, highlighting the influence of memory buffers and current tasks.

Contribution

It introduces unified hypothesis-based and prediction-based bounds for replay-based CL, providing tighter, computationally feasible generalization error estimates.

Findings

Bounds effectively capture generalization dynamics in CL

Prediction-based bounds are tighter and more tractable

Framework applies broadly to various learning algorithms

Abstract

Continual learning (CL) has emerged as a dominant paradigm for acquiring knowledge from sequential tasks while avoiding catastrophic forgetting. Although many CL methods have been proposed to show impressive empirical performance, the theoretical understanding of their generalization behavior remains limited, particularly for replay-based approaches. This paper establishes a unified theoretical framework for replay-based CL, deriving a series of information-theoretic generalization bounds that explicitly elucidate the impact of the memory buffer alongside the current task on generalization performance. Specifically, our hypothesis-based bounds capture the trade-off between the number of selected exemplars and the information dependency between the hypothesis and the memory buffer. Our prediction-based bounds yield tighter and computationally tractable upper bounds on the generalization error by leveraging low-dimensional variables. Theoretical analysis is general and broadly applicable to a wide range of learning algorithms, exemplified by stochastic gradient Langevin dynamics (SGLD) as a representative method. Comprehensive experimental evaluations demonstrate the effectiveness of our derived bounds in capturing the generalization dynamics in replay-based CL settings.