OccCylindrical: Multi-Modal Fusion with Cylindrical Representation for 3D Semantic Occupancy Prediction

TL;DR

OccCylindrical introduces a novel cylindrical coordinate-based multi-modal fusion approach for 3D semantic occupancy prediction, improving detail preservation and achieving state-of-the-art results in autonomous vehicle perception tasks.

Contribution

The paper proposes a cylindrical coordinate-based fusion method that enhances 3D semantic occupancy prediction by preserving geometric details better than Cartesian-based methods.

Findings

Achieves state-of-the-art performance on nuScenes dataset.

Effective in challenging rainy and nighttime conditions.

Outperforms existing multisensor fusion approaches.

Abstract

The safe operation of autonomous vehicles (AVs) is highly dependent on their understanding of the surroundings. For this, the task of 3D semantic occupancy prediction divides the space around the sensors into voxels, and labels each voxel with both occupancy and semantic information. Recent perception models have used multisensor fusion to perform this task. However, existing multisensor fusion-based approaches focus mainly on using sensor information in the Cartesian coordinate system. This ignores the distribution of the sensor readings, leading to a loss of fine-grained details and performance degradation. In this paper, we propose OccCylindrical that merges and refines the different modality features under cylindrical coordinates. Our method preserves more fine-grained geometry detail that leads to better performance. Extensive experiments conducted on the nuScenes dataset, including challenging rainy and nighttime scenarios, confirm our approach's effectiveness and state-of-the-art performance. The code will be available at: https://github.com/DanielMing123/OccCylindrical