Towards Open-Set Object Detection and Discovery

TL;DR

This paper introduces a new task, OSODD, enabling open-set object detectors to discover and categorize unknown objects without human labeling, advancing autonomous understanding of dynamic environments.

Contribution

It proposes a two-stage method for open-set detection and unsupervised discovery of new object categories, extending current open-set detection capabilities.

Findings

Effective detection of known and unknown objects on MS-COCO

Unsupervised discovery of novel object categories

Minimal supervision required for category definition

Abstract

With the human pursuit of knowledge, open-set object detection (OSOD) has been designed to identify unknown objects in a dynamic world. However, an issue with the current setting is that all the predicted unknown objects share the same category as "unknown", which require incremental learning via a human-in-the-loop approach to label novel classes. In order to address this problem, we present a new task, namely Open-Set Object Detection and Discovery (OSODD). This new task aims to extend the ability of open-set object detectors to further discover the categories of unknown objects based on their visual appearance without human effort. We propose a two-stage method that first uses an open-set object detector to predict both known and unknown objects. Then, we study the representation of predicted objects in an unsupervised manner and discover new categories from the set of unknown objects. With this method, a detector is able to detect objects belonging to known classes and define novel categories for objects of unknown classes with minimal supervision. We show the performance of our model on the MS-COCO dataset under a thorough evaluation protocol. We hope that our work will promote further research towards a more robust real-world detection system.