Learnability and Algorithm for Continual Learning

TL;DR

This paper proves that Class Incremental Learning (CIL) is learnable, introduces a new algorithm based on theoretical insights, and demonstrates its effectiveness through experiments.

Contribution

It establishes the learnability of CIL and proposes a novel algorithm grounded in theoretical analysis.

Findings

CIL is proven to be learnable.

The proposed algorithm outperforms existing methods.

Experimental results confirm the effectiveness of the new approach.

Abstract

This paper studies the challenging continual learning (CL) setting of Class Incremental Learning (CIL). CIL learns a sequence of tasks consisting of disjoint sets of concepts or classes. At any time, a single model is built that can be applied to predict/classify test instances of any classes learned thus far without providing any task related information for each test instance. Although many techniques have been proposed for CIL, they are mostly empirical. It has been shown recently that a strong CIL system needs a strong within-task prediction (WP) and a strong out-of-distribution (OOD) detection for each task. However, it is still not known whether CIL is actually learnable. This paper shows that CIL is learnable. Based on the theory, a new CIL algorithm is also proposed. Experimental results demonstrate its effectiveness.