Towards Flexible 3D Perception: Object-Centric Occupancy Completion Augments 3D Object Detection

TL;DR

This paper introduces object-centric occupancy as a detailed, high-resolution 3D representation to improve object detection and shape completion in autonomous driving, leveraging a new dataset and a novel neural network.

Contribution

It presents the first object-centric occupancy dataset and a new completion network that enhances 3D perception by providing detailed object shapes and improving detection accuracy.

Findings

Occupancy features improve detection of distant objects.

The proposed network accurately completes object shapes from noisy data.

The dataset enables training and evaluation of object-centric occupancy models.

Abstract

While 3D object bounding box (bbox) representation has been widely used in autonomous driving perception, it lacks the ability to capture the precise details of an object's intrinsic geometry. Recently, occupancy has emerged as a promising alternative for 3D scene perception. However, constructing a high-resolution occupancy map remains infeasible for large scenes due to computational constraints. Recognizing that foreground objects only occupy a small portion of the scene, we introduce object-centric occupancy as a supplement to object bboxes. This representation not only provides intricate details for detected objects but also enables higher voxel resolution in practical applications. We advance the development of object-centric occupancy perception from both data and algorithm perspectives. On the data side, we construct the first object-centric occupancy dataset from scratch using an automated pipeline. From the algorithmic standpoint, we introduce a novel object-centric occupancy completion network equipped with an implicit shape decoder that manages dynamic-size occupancy generation. This network accurately predicts the complete object-centric occupancy volume for inaccurate object proposals by leveraging temporal information from long sequences. Our method demonstrates robust performance in completing object shapes under noisy detection and tracking conditions. Additionally, we show that our occupancy features significantly enhance the detection results of state-of-the-art 3D object detectors, especially for incomplete or distant objects in the Waymo Open Dataset.