DuET: Dual Incremental Object Detection via Exemplar-Free Task Arithmetic

TL;DR

DuET is a novel exemplar-free framework for dual incremental object detection that effectively handles class and domain shifts simultaneously, improving retention and adaptability in real-world applications.

Contribution

We introduce DuET, a detector-agnostic task arithmetic framework with a Directional Consistency Loss for stable dual incremental learning without exemplars.

Findings

Achieves +13.12% RAI on Pascal Series with 89.3% Avg RI

Achieves +11.39% RAI on Diverse Weather Series with 88.57% Avg RI

Outperforms existing methods in dual incremental object detection

Abstract

Real-world object detection systems, such as those in autonomous driving and surveillance, must continuously learn new object categories and simultaneously adapt to changing environmental conditions. Existing approaches, Class Incremental Object Detection (CIOD) and Domain Incremental Object Detection (DIOD) only address one aspect of this challenge. CIOD struggles in unseen domains, while DIOD suffers from catastrophic forgetting when learning new classes, limiting their real-world applicability. To overcome these limitations, we introduce Dual Incremental Object Detection (DuIOD), a more practical setting that simultaneously handles class and domain shifts in an exemplar-free manner. We propose DuET, a Task Arithmetic-based model merging framework that enables stable incremental learning while mitigating sign conflicts through a novel Directional Consistency Loss. Unlike prior methods, DuET is detector-agnostic, allowing models like YOLO11 and RT-DETR to function as real-time incremental object detectors. To comprehensively evaluate both retention and adaptation, we introduce the Retention-Adaptability Index (RAI), which combines the Average Retention Index (Avg RI) for catastrophic forgetting and the Average Generalization Index for domain adaptability into a common ground. Extensive experiments on the Pascal Series and Diverse Weather Series demonstrate DuET's effectiveness, achieving a +13.12% RAI improvement while preserving 89.3% Avg RI on the Pascal Series (4 tasks), as well as a +11.39% RAI improvement with 88.57% Avg RI on the Diverse Weather Series (3 tasks), outperforming existing methods.