iCaRL: Incremental Classifier and Representation Learning

TL;DR

iCaRL introduces a novel incremental learning method that enables deep neural networks to learn new classes over time without forgetting previous ones, using a combined classifier and representation learning approach.

Contribution

The paper presents iCaRL, a new training strategy for class-incremental learning that integrates classifier and representation learning, overcoming limitations of fixed representations.

Findings

iCaRL effectively learns many classes incrementally on CIFAR-100 and ImageNet.

It outperforms previous methods that struggle with long-term incremental learning.

iCaRL maintains high accuracy over extended class addition periods.

Abstract

A major open problem on the road to artificial intelligence is the development of incrementally learning systems that learn about more and more concepts over time from a stream of data. In this work, we introduce a new training strategy, iCaRL, that allows learning in such a class-incremental way: only the training data for a small number of classes has to be present at the same time and new classes can be added progressively. iCaRL learns strong classifiers and a data representation simultaneously. This distinguishes it from earlier works that were fundamentally limited to fixed data representations and therefore incompatible with deep learning architectures. We show by experiments on CIFAR-100 and ImageNet ILSVRC 2012 data that iCaRL can learn many classes incrementally over a long period of time where other strategies quickly fail.