Semantic Scene Completion Combining Colour and Depth: preliminary experiments

TL;DR

This paper explores enhancing semantic scene completion by integrating colour information with depth data in a 3D convolutional network, aiming to improve scene understanding from single-view observations.

Contribution

It investigates the use of RGB colour channels alongside depth maps to improve the performance of SSCnet in semantic scene completion tasks.

Findings

Colour information can potentially enhance scene completion accuracy.

Preliminary experiments suggest benefits of combining colour and depth data.

Further research is needed to quantify improvements.

Abstract

Semantic scene completion is the task of producing a complete 3D voxel representation of volumetric occupancy with semantic labels for a scene from a single-view observation. We built upon the recent work of Song et al. (CVPR 2017), who proposed SSCnet, a method that performs scene completion and semantic labelling in a single end-to-end 3D convolutional network. SSCnet uses only depth maps as input, even though depth maps are usually obtained from devices that also capture colour information, such as RGBD sensors and stereo cameras. In this work, we investigate the potential of the RGB colour channels to improve SSCnet.