Beyond Known Objects: A Novel Framework for Open-Set Object Detection using Negative-Aware Norm

TL;DR

NAN-SPOT is a training-light open-set object detection framework that leverages existing detectors and a new metric to identify unknown objects efficiently, without extensive retraining.

Contribution

It introduces NAN-SPOT, a novel method that estimates objectness using a hidden layer metric, eliminating the need for retraining base detectors in open-set detection.

Findings

NAN-SPOT outperforms heavily trained methods in unknown object detection.

It requires only minutes of training on a small dataset.

It improves detection performance on an expanded COCO-Open dataset.

Abstract

Open-Set Object Detection (OSOD) is crucial for autonomous driving, where perception systems must recognize and localize both known and previously unseen objects in complex, dynamic environments. While recent approaches deliver promising results, they often require retraining the detector extensively to learn objectness, which describes the likelihood that a bounding box tightly encloses a valid object, regardless of whether its category was learned during training. Deviating from existing work, we hypothesize that standard off-the-shelf detectors may already contain helpful cues for objectness, owing to their training on numerous and diverse known categories. Building on this idea, we propose NAN-SPOT, a training-light framework that does not require to retrain the base object detector and estimates objectness by leveraging a hidden layer metric called Negative-Aware Norm (NAN), requiring only minutes of training on just hundreds of images. To support comprehensive evaluation, we introduce COCO-Open, an expanded version of the existing COCO-Mixed dataset, increasing unknown object annotations from 433 to 1853, making it the most exhaustively labeled dataset for OSOD to the best of our knowledge. Experimental results demonstrate that NAN-SPOT achieves even better performance on unknown object detection than methods requiring heavy training, without compromising performance on known objects. This efficiency and robustness make NAN-SPOT a promising step towards open-world perception in autonomous driving.