Knowledge Consolidation based Class Incremental Online Learning with Limited Data

TL;DR

This paper introduces a novel meta-learning and knowledge consolidation approach for class incremental online learning with limited data, effectively addressing catastrophic forgetting and overfitting without replay memory.

Contribution

It proposes a new method combining meta-learning and knowledge consolidation to improve online class incremental learning with scarce data and no replay memory.

Findings

Outperforms existing methods on multiple benchmarks.

Effectively mitigates catastrophic forgetting.

Learns robust, generalizable representations.

Abstract

We propose a novel approach for class incremental online learning in a limited data setting. This problem setting is challenging because of the following constraints: (1) Classes are given incrementally, which necessitates a class incremental learning approach; (2) Data for each class is given in an online fashion, i.e., each training example is seen only once during training; (3) Each class has very few training examples; and (4) We do not use or assume access to any replay/memory to store data from previous classes. Therefore, in this setting, we have to handle twofold problems of catastrophic forgetting and overfitting. In our approach, we learn robust representations that are generalizable across tasks without suffering from the problems of catastrophic forgetting and overfitting to accommodate future classes with limited samples. Our proposed method leverages the meta-learning framework with knowledge consolidation. The meta-learning framework helps the model for rapid learning when samples appear in an online fashion. Simultaneously, knowledge consolidation helps to learn a robust representation against forgetting under online updates to facilitate future learning. Our approach significantly outperforms other methods on several benchmarks.