DeeSIL: Deep-Shallow Incremental Learning

TL;DR

DeeSIL is an incremental learning method that uses a fixed deep feature extractor and shallow classifiers, enabling fast updates and effective learning with limited memory, outperforming existing methods on ImageNet.

Contribution

DeeSIL introduces a transfer learning-based incremental learning scheme that avoids deep retraining, allowing rapid addition of new classes with limited memory.

Findings

DeeSIL outperforms three state-of-the-art algorithms on ImageNet.

It achieves 23-33 points higher accuracy with the same or more initial data.

DeeSIL adds new classes within a minute without deep retraining.

Abstract

Incremental Learning (IL) is an interesting AI problem when the algorithm is assumed to work on a budget. This is especially true when IL is modeled using a deep learning approach, where two com- plex challenges arise due to limited memory, which induces catastrophic forgetting and delays related to the retraining needed in order to incorpo- rate new classes. Here we introduce DeeSIL, an adaptation of a known transfer learning scheme that combines a fixed deep representation used as feature extractor and learning independent shallow classifiers to in- crease recognition capacity. This scheme tackles the two aforementioned challenges since it works well with a limited memory budget and each new concept can be added within a minute. Moreover, since no deep re- training is needed when the model is incremented, DeeSIL can integrate larger amounts of initial data that provide more transferable features. Performance is evaluated on ImageNet LSVRC 2012 against three state of the art algorithms. Results show that, at scale, DeeSIL performance is 23 and 33 points higher than the best baseline when using the same and more initial data respectively.