Behind the Curtain: Learning Occluded Shapes for 3D Object Detection

TL;DR

This paper introduces BtcDet, a novel LiDAR-based 3D object detection model that learns object shape priors to accurately detect partially occluded objects in point clouds, significantly improving detection performance.

Contribution

The paper proposes a new model that predicts object shape priors and occupancy probabilities to handle occlusion in LiDAR point clouds, advancing 3D detection accuracy.

Findings

BtcDet outperforms state-of-the-art methods on KITTI and Waymo datasets.

It effectively predicts complete object shapes from partial observations.

The approach improves detection of occluded objects like cars and cyclists.

Abstract

Advances in LiDAR sensors provide rich 3D data that supports 3D scene understanding. However, due to occlusion and signal miss, LiDAR point clouds are in practice 2.5D as they cover only partial underlying shapes, which poses a fundamental challenge to 3D perception. To tackle the challenge, we present a novel LiDAR-based 3D object detection model, dubbed Behind the Curtain Detector (BtcDet), which learns the object shape priors and estimates the complete object shapes that are partially occluded (curtained) in point clouds. BtcDet first identifies the regions that are affected by occlusion and signal miss. In these regions, our model predicts the probability of occupancy that indicates if a region contains object shapes. Integrated with this probability map, BtcDet can generate high-quality 3D proposals. Finally, the probability of occupancy is also integrated into a proposal refinement module to generate the final bounding boxes. Extensive experiments on the KITTI Dataset and the Waymo Open Dataset demonstrate the effectiveness of BtcDet. Particularly, for the 3D detection of both cars and cyclists on the KITTI benchmark, BtcDet surpasses all of the published state-of-the-art methods by remarkable margins. Code is released (https://github.com/Xharlie/BtcDet}{https://github.com/Xharlie/BtcDet).