HD$^2$-SSC: High-Dimension High-Density Semantic Scene Completion for Autonomous Driving

TL;DR

This paper introduces HD$^2$-SSC, a novel framework for 3D semantic scene completion in autonomous driving that addresses the dimension and density gaps in current methods, leading to more accurate dense occupancy predictions.

Contribution

The paper proposes the HD$^2$-SSC framework with modules for expanding pixel semantics and refining voxel occupancy, effectively bridging the dimension and density gaps in scene completion.

Findings

Outperforms existing methods on SemanticKITTI and SSCBench-KITTI-360 datasets.

Effectively bridges the dimension and density gaps in 3D scene completion.

Enhances semantic density and accuracy of voxel occupancy predictions.

Abstract

Camera-based 3D semantic scene completion (SSC) plays a crucial role in autonomous driving, enabling voxelized 3D scene understanding for effective scene perception and decision-making. Existing SSC methods have shown efficacy in improving 3D scene representations, but suffer from the inherent input-output dimension gap and annotation-reality density gap, where the 2D planner view from input images with sparse annotated labels leads to inferior prediction of real-world dense occupancy with a 3D stereoscopic view. In light of this, we propose the corresponding High-Dimension High-Density Semantic Scene Completion (HD$^2$-SSC) framework with expanded pixel semantics and refined voxel occupancies. To bridge the dimension gap, a High-dimension Semantic Decoupling module is designed to expand 2D image features along a pseudo third dimension, decoupling coarse pixel semantics from occlusions, and then identify focal regions with fine semantics to enrich image features. To mitigate the density gap, a High-density Occupancy Refinement module is devised with a "detect-and-refine" architecture to leverage contextual geometric and semantic structures for enhanced semantic density with the completion of missing voxels and correction of erroneous ones. Extensive experiments and analyses on the SemanticKITTI and SSCBench-KITTI-360 datasets validate the effectiveness of our HD$^2$-SSC framework.