Lightweight Spatial Embedding for Vision-based 3D Occupancy Prediction

TL;DR

LightOcc introduces a lightweight spatial embedding framework that enhances 3D occupancy prediction from 2D BEV features, achieving state-of-the-art accuracy with reduced computational overhead for real-time autonomous driving.

Contribution

The paper proposes a novel Lightweight Spatial Embedding method that effectively supplements height information in BEV features, improving accuracy and efficiency in 3D occupancy prediction.

Findings

Significantly improves occupancy prediction accuracy.

Achieves state-of-the-art performance on Occ3D-nuScenes.

Reduces computational and memory overhead.

Abstract

Occupancy prediction has garnered increasing attention in recent years for its comprehensive fine-grained environmental representation and strong generalization to open-set objects. However, cumbersome voxel features and 3D convolution operations inevitably introduce large overheads in both memory and computation, obstructing the deployment of occupancy prediction approaches in real-time autonomous driving systems. Although some methods attempt to efficiently predict 3D occupancy from 2D Bird's-Eye-View (BEV) features through the Channel-to-Height mechanism, BEV features are insufficient to store all the height information of the scene, which limits performance. This paper proposes LightOcc, an innovative 3D occupancy prediction framework that leverages Lightweight Spatial Embedding to effectively supplement the height clues for the BEV-based representation while maintaining its deployability. Firstly, Global Spatial Sampling is used to obtain the Single-Channel Occupancy from multi-view depth distribution. Spatial-to-Channel mechanism then takes the arbitrary spatial dimension of Single-Channel Occupancy as the feature dimension and extracts Tri-Perspective Views (TPV) Embeddings by 2D convolution. Finally, TPV Embeddings will interact with each other by Lightweight TPV Interaction module to obtain the Spatial Embedding that is optimal supplementary to BEV features. Sufficient experimental results show that LightOcc significantly increases the prediction accuracy of the baseline and achieves state-of-the-art performance on the Occ3D-nuScenes benchmark.