Disentangling Instance and Scene Contexts for 3D Semantic Scene Completion

TL;DR

This paper introduces DISC, a dual-stream approach for 3D Semantic Scene Completion that disentangles instance and scene contexts, leading to state-of-the-art results by leveraging class-specific information and priors.

Contribution

DISC proposes a novel dual-stream paradigm with class queries and specialized decoding modules to improve 3D scene completion and instance segmentation performance.

Findings

Achieves SOTA mIoU scores of 17.35 on SemanticKITTI and 20.55 on SSCBench-KITTI-360.

Outperforms multi-frame methods using only single-frame input.

Significantly improves instance category performance, surpassing previous SOTA by 17.9% and 11.9%.

Abstract

3D Semantic Scene Completion (SSC) has gained increasing attention due to its pivotal role in 3D perception. Recent advancements have primarily focused on refining voxel-level features to construct 3D scenes. However, treating voxels as the basic interaction units inherently limits the utilization of class-level information, which is proven critical for enhancing the granularity of completion results. To address this, we propose \textbf{D}isentangling Instance and Scene Contexts (DISC), a novel dual-stream paradigm that enhances learning for both instance and scene categories through separated optimization. Specifically, we replace voxel queries with discriminative class queries, which incorporate class-specific geometric and semantic priors. Additionally, we exploit the intrinsic properties of classes to design specialized decoding modules, facilitating targeted interactions and efficient class-level information flow. Experimental results demonstrate that DISC achieves state-of-the-art (SOTA) performance on both SemanticKITTI and SSCBench-KITTI-360 benchmarks, with mIoU scores of 17.35 and 20.55, respectively. Remarkably, DISC even outperforms multi-frame SOTA methods using only single-frame input and significantly improves instance category performance, surpassing both single-frame and multi-frame SOTA instance mIoU by 17.9\% and 11.9\%, respectively, on the SemanticKITTI hidden test. The code is available at https://github.com/Enyu-Liu/DISC.