BePo: Dual Representation for 3D Occupancy Prediction

TL;DR

BePo introduces a dual representation approach combining BEV and sparse points with cross-attention to improve 3D occupancy prediction efficiency and accuracy in autonomous driving scenarios.

Contribution

The paper proposes BePo, a novel dual representation method that effectively combines BEV and sparse points for better 3D occupancy prediction with low inference cost.

Findings

BePo outperforms existing methods on multiple benchmarks.

BePo achieves high accuracy with reduced computational costs.

The dual representation improves detection of small and large objects.

Abstract

3D occupancy infers fine-grained 3D geometry and semantics which is critical for autonomous driving. Most existing approaches carry high compute costs, requiring dense 3D feature volume and cross-attention to effectively aggregate information. More efficient methods adopt Bird's Eye View (BEV) or sparse points as scene representation leading to much reduced runtime. However, BEV struggles with small objects that often have very limited feature representation especially after being projected to the ground plane. Sparse points on the other and, can model objects of various sizes in 3D space, but is inefficient at capturing flat surfaces or large objects. To address these shortcomings, we present BePo, which features a dual representation of BEV and sparse points. The 3D information learned in the sparse points branch is shared with the BEV stream via cross-attention, which injects learning signals of difficult objects on the BEV plane. The outputs of both branches are then fused to generate the final 3D occupancy predictions. Extensive experiments on a suite of challenging benchmarks including Occ3D-nuScenes, Occ3D-Waymo and Occ-ScanNet demonstrate the superiority of our proposed BePo. In addition, BePo carries low inference cost even when compared to latest efficient methods.