One Step Closer: Creating the Future to Boost Monocular Semantic Scene Completion

TL;DR

This paper introduces CF-SSC, a temporal framework for monocular 3D semantic scene completion that predicts future frames to improve occlusion handling and scene understanding in autonomous driving scenarios.

Contribution

It proposes a novel pseudo-future frame prediction method combined with 3D-aware architecture for enhanced scene completion from monocular images.

Findings

Achieves state-of-the-art results on SemanticKITTI and SSCBench-KITTI-360 benchmarks.

Effectively improves occlusion reasoning and 3D scene understanding.

Demonstrates robustness in real-world autonomous driving scenarios.

Abstract

In recent years, visual 3D Semantic Scene Completion (SSC) has emerged as a critical perception task for autonomous driving due to its ability to infer complete 3D scene layouts and semantics from single 2D images. However, in real-world traffic scenarios, a significant portion of the scene remains occluded or outside the camera's field of view -- a fundamental challenge that existing monocular SSC methods fail to address adequately. To overcome these limitations, we propose Creating the Future SSC (CF-SSC), a novel temporal SSC framework that leverages pseudo-future frame prediction to expand the model's effective perceptual range. Our approach combines poses and depths to establish accurate 3D correspondences, enabling geometrically-consistent fusion of past, present, and predicted future frames in 3D space. Unlike conventional methods that rely on simple feature stacking, our 3D-aware architecture achieves more robust scene completion by explicitly modeling spatial-temporal relationships. Comprehensive experiments on SemanticKITTI and SSCBench-KITTI-360 benchmarks demonstrate state-of-the-art performance, validating the effectiveness of our approach, highlighting our method's ability to improve occlusion reasoning and 3D scene completion accuracy.