Incremental Learning of Object Detectors without Catastrophic Forgetting

TL;DR

This paper introduces a method for incremental object detection that prevents catastrophic forgetting by using a specialized loss function, enabling models to learn new classes without access to original training data.

Contribution

The paper proposes a novel loss function combining detection and distillation losses to enable incremental learning of object detectors without catastrophic forgetting.

Findings

Effective in preventing performance degradation on old classes

Allows multiple incremental learning steps with moderate performance loss

Demonstrated on PASCAL VOC 2007 and COCO datasets

Abstract

Despite their success for object detection, convolutional neural networks are ill-equipped for incremental learning, i.e., adapting the original model trained on a set of classes to additionally detect objects of new classes, in the absence of the initial training data. They suffer from "catastrophic forgetting" - an abrupt degradation of performance on the original set of classes, when the training objective is adapted to the new classes. We present a method to address this issue, and learn object detectors incrementally, when neither the original training data nor annotations for the original classes in the new training set are available. The core of our proposed solution is a loss function to balance the interplay between predictions on the new classes and a new distillation loss which minimizes the discrepancy between responses for old classes from the original and the updated networks. This incremental learning can be performed multiple times, for a new set of classes in each step, with a moderate drop in performance compared to the baseline network trained on the ensemble of data. We present object detection results on the PASCAL VOC 2007 and COCO datasets, along with a detailed empirical analysis of the approach.