Continual Learning by Three-Phase Consolidation

TL;DR

This paper introduces TPC, a three-phase method for continual learning that reduces forgetting and class bias, demonstrating improved accuracy and efficiency on complex datasets with reproducible results.

Contribution

The paper proposes a novel three-phase consolidation approach for continual learning that effectively manages class bias and forgetting, with comprehensive experiments validating its advantages.

Findings

Outperforms existing methods in accuracy on complex datasets

Reduces forgetting of previous knowledge during learning

Provides fully reproducible results within the Avalanche framework

Abstract

TPC (Three-Phase Consolidation) is here introduced as a simple but effective approach to continually learn new classes (and/or instances of known classes) while controlling forgetting of previous knowledge. Each experience (a.k.a. task) is learned in three phases characterized by different rules and learning dynamics, aimed at removing the class-bias problem (due to class unbalancing) and limiting gradient-based corrections to prevent forgetting of underrepresented classes. Several experiments on complex datasets demonstrate its accuracy and efficiency advantages over competitive existing approaches. The algorithm and all the results presented in this paper are fully reproducible thanks to its publication on the Avalanche open framework for continual learning.