Towards Adaptive Open-Set Object Detection via Category-Level Collaboration Knowledge Mining

TL;DR

This paper introduces a novel category-level collaboration knowledge mining approach to improve adaptive open-set object detection, effectively handling domain shifts and novel categories without target annotations.

Contribution

It proposes a clustering-based memory bank and adaptive feature strategies to enhance cross-domain representations and reduce source bias in AOOD.

Findings

Outperforms state-of-the-art AOOD methods by 1.1-5.5 mAP on multiple benchmarks.

Uses a clustering-based memory bank to encode class prototypes and intra-class disparities.

Employs a base-to-novel selection metric and adaptive feature assignment to improve detection of novel categories.

Abstract

Existing object detectors often struggle to generalize across domains while adapting to emerging novel categories. Adaptive open-set object detection (AOOD) addresses this challenge by training on base categories in the source domain and adapting to both base and novel categories in the target domain without target annotations. However, current AOOD methods remain limited by weak cross-domain representations, ambiguity among novel categories, and source-domain feature bias. To address these issues, we propose a category-level collaboration knowledge mining strategy that exploits both inter-class and intra-class relationships across domains. Specifically, we construct a clustering-based memory bank to encode class prototypes, auxiliary features, and intra-class disparity information, and iteratively update it via unsupervised clustering to enhance category-level knowledge representation. We further design a base-to-novel selection metric to discover source-domain features related to novel categories and use them to initialize novel-category classifiers. In addition, an adaptive feature assignment strategy transfers the learned category-level knowledge to the target domain and asynchronously updates the memory bank to alleviate source-domain bias. Extensive experiments on multiple benchmarks show that our method consistently surpasses state-of-the-art AOOD methods by 1.1-5.5 mAP.