Open-set 3D Object Detection

TL;DR

This paper introduces MLUC, an open-set 3D object detection method that accurately identifies known and unknown objects by leveraging metric learning and unsupervised clustering, improving robustness in practical robot perception systems.

Contribution

The paper proposes a novel open-set 3D object detection approach combining metric learning and clustering, addressing the challenge of unknown object identification.

Findings

Achieves state-of-the-art performance in open-set detection

Effectively differentiates known and unknown objects using Euclidean distance-based confidence

Successfully encloses unknown objects with accurate bounding boxes

Abstract

3D object detection has been wildly studied in recent years, especially for robot perception systems. However, existing 3D object detection is under a closed-set condition, meaning that the network can only output boxes of trained classes. Unfortunately, this closed-set condition is not robust enough for practical use, as it will identify unknown objects as known by mistake. Therefore, in this paper, we propose an open-set 3D object detector, which aims to (1) identify known objects, like the closed-set detection, and (2) identify unknown objects and give their accurate bounding boxes. Specifically, we divide the open-set 3D object detection problem into two steps: (1) finding out the regions containing the unknown objects with high probability and (2) enclosing the points of these regions with proper bounding boxes. The first step is solved by the finding that unknown objects are often classified as known objects with low confidence, and we show that the Euclidean distance sum based on metric learning is a better confidence score than the naive softmax probability to differentiate unknown objects from known objects. On this basis, unsupervised clustering is used to refine the bounding boxes of unknown objects. The proposed method combining metric learning and unsupervised clustering is called the MLUC network. Our experiments show that our MLUC network achieves state-of-the-art performance and can identify both known and unknown objects as expected.