Incremental Object Detection with Prompt-based Methods

TL;DR

This paper investigates the application of prompt-based methods to incremental object detection, revealing their limitations and proposing a combined approach with data replay that improves performance.

Contribution

It is the first to analyze prompt-based methods in incremental object detection and demonstrates that combining prompts with data replay enhances results.

Findings

Prompt-based methods underperform in incremental object detection.

Combining visual prompts with data replay yields the best performance.

Insights on prompt length and initialization inform future research.

Abstract

Visual prompt-based methods have seen growing interest in incremental learning (IL) for image classification. These approaches learn additional embedding vectors while keeping the model frozen, making them efficient to train. However, no prior work has applied such methods to incremental object detection (IOD), leaving their generalizability unclear. In this paper, we analyze three different prompt-based methods under a complex domain-incremental learning setting. We additionally provide a wide range of reference baselines for comparison. Empirically, we show that the prompt-based approaches we tested underperform in this setting. However, a strong yet practical method, combining visual prompts with replaying a small portion of previous data, achieves the best results. Together with additional experiments on prompt length and initialization, our findings offer valuable insights for advancing prompt-based IL in IOD.