SOGDet: Semantic-Occupancy Guided Multi-view 3D Object Detection

TL;DR

SOGDet introduces a 3D semantic-occupancy branch to enhance multi-view 3D object detection, improving accuracy by incorporating environmental context in autonomous driving scenarios.

Contribution

The paper presents a novel semantic-occupancy guided approach that can be integrated with existing BEV-based methods to improve 3D detection performance.

Findings

Consistently improves baseline methods on nuScenes dataset.

Enhances detection scores such as NDS and mAP.

Leverages environmental context for more robust perception.

Abstract

In the field of autonomous driving, accurate and comprehensive perception of the 3D environment is crucial. Bird's Eye View (BEV) based methods have emerged as a promising solution for 3D object detection using multi-view images as input. However, existing 3D object detection methods often ignore the physical context in the environment, such as sidewalk and vegetation, resulting in sub-optimal performance. In this paper, we propose a novel approach called SOGDet (Semantic-Occupancy Guided Multi-view 3D Object Detection), that leverages a 3D semantic-occupancy branch to improve the accuracy of 3D object detection. In particular, the physical context modeled by semantic occupancy helps the detector to perceive the scenes in a more holistic view. Our SOGDet is flexible to use and can be seamlessly integrated with most existing BEV-based methods. To evaluate its effectiveness, we apply this approach to several state-of-the-art baselines and conduct extensive experiments on the exclusive nuScenes dataset. Our results show that SOGDet consistently enhance the performance of three baseline methods in terms of nuScenes Detection Score (NDS) and mean Average Precision (mAP). This indicates that the combination of 3D object detection and 3D semantic occupancy leads to a more comprehensive perception of the 3D environment, thereby aiding build more robust autonomous driving systems. The codes are available at: https://github.com/zhouqiu/SOGDet.