RICO: Two Realistic Benchmarks and an In-Depth Analysis for Incremental Learning in Object Detection

TL;DR

This paper introduces two realistic benchmarks for incremental object detection, highlighting the challenges and limitations of current methods in real-world scenarios with diverse data and domain shifts.

Contribution

The paper presents RICO benchmarks based on diverse datasets to evaluate incremental learning in object detection under realistic conditions.

Findings

All IL methods underperform in adaptability and retention.

Replaying a small amount of previous data outperforms existing methods.

Individual training on combined data remains superior.

Abstract

Incremental Learning (IL) trains models sequentially on new data without full retraining, offering privacy, efficiency, and scalability. IL must balance adaptability to new data with retention of old knowledge. However, evaluations often rely on synthetic, simplified benchmarks, obscuring real-world IL performance. To address this, we introduce two Realistic Incremental Object Detection Benchmarks (RICO): Domain RICO (D-RICO) features domain shifts with a fixed class set, and Expanding-Classes RICO (EC-RICO) integrates new domains and classes per IL step. Built from 14 diverse datasets covering real and synthetic domains, varying conditions (e.g., weather, time of day), camera sensors, perspectives, and labeling policies, both benchmarks capture challenges absent in existing evaluations. Our experiments show that all IL methods underperform in adaptability and retention, while replaying a small amount of previous data already outperforms all methods. However, individual training on the data remains superior. We heuristically attribute this gap to weak teachers in distillation, single models' inability to manage diverse tasks, and insufficient plasticity. Our code will be made publicly available.