Dr.Occ: Depth- and Region-Guided 3D Occupancy from Surround-View Cameras for Autonomous Driving

TL;DR

Dr. Occ introduces a novel framework for 3D occupancy prediction in autonomous driving that combines depth-guided view transformation and region-specific experts to improve geometric and semantic accuracy.

Contribution

It proposes a depth- and region-guided occupancy prediction framework with a depth-guided view transformer and a region-specific expert transformer, addressing geometric misalignment and spatial class imbalance.

Findings

Improves BEVDet4D baseline by 7.43% mIoU on Occ3D--nuScenes.

Enhances geometric alignment with depth guidance.

Addresses spatial semantic variations with region-specific experts.

Abstract

3D semantic occupancy prediction is crucial for autonomous driving perception, offering comprehensive geometric scene understanding and semantic recognition. However, existing methods struggle with geometric misalignment in view transformation due to the lack of pixel-level accurate depth estimation, and severe spatial class imbalance where semantic categories exhibit strong spatial anisotropy. To address these challenges, we propose Dr. Occ, a depth- and region-guided occupancy prediction framework. Specifically, we introduce a depth-guided 2D-to-3D View Transformer (D$^2$-VFormer) that effectively leverages high-quality dense depth cues from MoGe-2 to construct reliable geometric priors, thereby enabling precise geometric alignment of voxel features. Moreover, inspired by the Mixture-of-Experts (MoE) framework, we propose a region-guided Expert Transformer (R/R$^2$-EFormer) that adaptively allocates region-specific experts to focus on different spatial regions, effectively addressing spatial semantic variations. Thus, the two components make complementary contributions: depth guidance ensures geometric alignment, while region experts enhance semantic learning. Experiments on the Occ3D--nuScenes benchmark demonstrate that Dr. Occ improves the strong baseline BEVDet4D by 7.43% mIoU and 3.09% IoU under the full vision-only setting.