Beyond Prompt Degradation: Prototype-guided Dual-pool Prompting for Incremental Object Detection

TL;DR

This paper introduces PDP, a prompt-decoupled framework with dual pools and pseudo-labeling to improve incremental object detection, significantly reducing prompt degradation and enhancing performance on benchmark datasets.

Contribution

The paper proposes a novel prompt-decoupled framework with dual pools and a pseudo-labeling module to address prompt degradation in incremental object detection.

Findings

Achieves 9.2% AP improvement on MS-COCO

Achieves 3.3% AP improvement on PASCAL VOC

Effectively mitigates prompt degradation and maintains supervision consistency

Abstract

Incremental Object Detection (IOD) aims to continuously learn new object categories without forgetting previously learned ones. Recently, prompt-based methods have gained popularity for their replay-free design and parameter efficiency. However, due to prompt coupling and prompt drift, these methods often suffer from prompt degradation during continual adaptation. To address these issues, we propose a novel prompt-decoupled framework called PDP. PDP innovatively designs a dual-pool prompt decoupling paradigm, which consists of a shared pool used to capture task-general knowledge for forward transfer, and a private pool used to learn task-specific discriminative features. This paradigm explicitly separates task-general and task-specific prompts, preventing interference between prompts and mitigating prompt coupling. In addition, to counteract prompt drift resulting from inconsistent supervision where old foreground objects are treated as background in subsequent tasks, PDP introduces a Prototypical Pseudo-Label Generation (PPG) module. PPG can dynamically update the class prototype space during training and use the class prototypes to further filter valuable pseudo-labels, maintaining supervisory signal consistency throughout the incremental process. PDP achieves state-of-the-art performance on MS-COCO (with a 9.2\% AP improvement) and PASCAL VOC (with a 3.3\% AP improvement) benchmarks, highlighting its potential in balancing stability and plasticity. The code and dataset are released at: https://github.com/zyt95579/PDP\_IOD/tree/main