Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detection

TL;DR

This paper introduces Augmented Box Replay, a novel method for incremental object detection that addresses foreground shift by replaying only foreground objects, reducing forgetting and storage needs.

Contribution

The paper proposes Augmented Box Replay, which stores and replays only foreground objects, and an Attentive RoI Distillation loss, advancing incremental object detection techniques.

Findings

Achieves state-of-the-art results on Pascal-VOC and COCO datasets.

Significantly reduces forgetting of previous classes.

Requires less storage than standard image replay.

Abstract

In incremental learning, replaying stored samples from previous tasks together with current task samples is one of the most efficient approaches to address catastrophic forgetting. However, unlike incremental classification, image replay has not been successfully applied to incremental object detection (IOD). In this paper, we identify the overlooked problem of foreground shift as the main reason for this. Foreground shift only occurs when replaying images of previous tasks and refers to the fact that their background might contain foreground objects of the current task. To overcome this problem, a novel and efficient Augmented Box Replay (ABR) method is developed that only stores and replays foreground objects and thereby circumvents the foreground shift problem. In addition, we propose an innovative Attentive RoI Distillation loss that uses spatial attention from region-of-interest (RoI) features to constrain current model to focus on the most important information from old model. ABR significantly reduces forgetting of previous classes while maintaining high plasticity in current classes. Moreover, it considerably reduces the storage requirements when compared to standard image replay. Comprehensive experiments on Pascal-VOC and COCO datasets support the state-of-the-art performance of our model.