DAOcc: 3D Object Detection Assisted Multi-Sensor Fusion for 3D Occupancy Prediction

TL;DR

DAOcc introduces a multi-modal 3D occupancy prediction framework that leverages 3D object detection supervision, employs a deployment-friendly backbone, and achieves state-of-the-art results with high efficiency on popular benchmarks.

Contribution

The paper proposes DAOcc, a novel multi-modal occupancy prediction method that effectively combines 3D object detection supervision with a practical, low-resolution input approach for improved performance.

Findings

Achieves state-of-the-art results on Occ3D-nuScenes and Occ3D-Waymo benchmarks.

Outperforms previous methods significantly with only ResNet-50 backbone and 256*704 input resolution.

Reaches 104.9 FPS with TensorRT optimization on an NVIDIA RTX 4090.

Abstract

Multi-sensor fusion significantly enhances the accuracy and robustness of 3D semantic occupancy prediction, which is crucial for autonomous driving and robotics. However, most existing approaches depend on high-resolution images and complex networks to achieve top performance, hindering their deployment in practical scenarios. Moreover, current multi-sensor fusion approaches mainly focus on improving feature fusion while largely neglecting effective supervision strategies for those features. To address these issues, we propose DAOcc, a novel multi-modal occupancy prediction framework that leverages 3D object detection supervision to assist in achieving superior performance, while using a deployment-friendly image backbone and practical input resolution. In addition, we introduce a BEV View Range Extension strategy to mitigate performance degradation caused by lower image resolution. Extensive experiments demonstrate that DAOcc achieves new state-of-the-art results on both the Occ3D-nuScenes and Occ3D-Waymo benchmarks, and outperforms previous state-of-the-art methods by a significant margin using only a ResNet-50 backbone and 256*704 input resolution. With TensorRT optimization, DAOcc reaches 104.9 FPS while maintaining 54.2 mIoU on an NVIDIA RTX 4090 GPU. Code is available at https://github.com/AlphaPlusTT/DAOcc.