iDPA: Instance Decoupled Prompt Attention for Incremental Medical Object Detection

TL;DR

This paper introduces iDPA, a novel framework for incremental medical object detection that decouples prompt generation and attention, leading to improved performance and reduced memory usage across multiple datasets.

Contribution

The paper proposes the iDPA framework with instance-level prompt generation and decoupled prompt attention, addressing challenges in incremental medical object detection.

Findings

iDPA outperforms state-of-the-art methods on 13 medical datasets.

Achieves significant improvements in FAP across various few-shot settings.

Reduces memory usage and mitigates catastrophic forgetting.

Abstract

Existing prompt-based approaches have demonstrated impressive performance in continual learning, leveraging pre-trained large-scale models for classification tasks; however, the tight coupling between foreground-background information and the coupled attention between prompts and image-text tokens present significant challenges in incremental medical object detection tasks, due to the conceptual gap between medical and natural domains. To overcome these challenges, we introduce the \method~framework, which comprises two main components: 1) Instance-level Prompt Generation (\ipg), which decouples fine-grained instance-level knowledge from images and generates prompts that focus on dense predictions, and 2) Decoupled Prompt Attention (\dpa), which decouples the original prompt attention, enabling a more direct and efficient transfer of prompt information while reducing memory usage and mitigating catastrophic forgetting. We collect 13 clinical, cross-modal, multi-organ, and multi-category datasets, referred to as \dataset, and experiments demonstrate that \method~outperforms existing SOTA methods, with FAP improvements of 5.44\%, 4.83\%, 12.88\%, and 4.59\% in full data, 1-shot, 10-shot, and 50-shot settings, respectively.