Move-to-Data: A new Continual Learning approach with Deep CNNs, Application for image-class recognition

TL;DR

This paper introduces Move-to-Data, a continual learning method for deep CNNs that efficiently updates models with new data, maintaining performance while reducing computational costs, demonstrated on CIFAR dataset.

Contribution

The paper proposes a novel fast continual learning layer for deep CNNs that allows efficient model updates with new data, reducing computational costs compared to retraining.

Findings

Achieves similar accuracy to retraining methods on CIFAR dataset.

Significantly lowers computational costs during model updates.

Effective for incremental learning in image classification tasks.

Abstract

In many real-life tasks of application of supervised learning approaches, all the training data are not available at the same time. The examples are lifelong image classification or recognition of environmental objects during interaction of instrumented persons with their environment, enrichment of an online-database with more images. It is necessary to pre-train the model at a "training recording phase" and then adjust it to the new coming data. This is the task of incremental/continual learning approaches. Amongst different problems to be solved by these approaches such as introduction of new categories in the model, refining existing categories to sub-categories and extending trained classifiers over them, ... we focus on the problem of adjusting pre-trained model with new additional training data for existing categories. We propose a fast continual learning layer at the end of the neuronal network. Obtained results are illustrated on the opensource CIFAR benchmark dataset. The proposed scheme yields similar performances as retraining but with drastically lower computational cost.