Monocular Spherical Depth Estimation with Explicitly Connected Weak Layout Cues

TL;DR

This paper introduces a novel approach that jointly estimates depth and room layout from single 360-degree images, leveraging a new dataset with automatic annotations to improve indoor scene understanding.

Contribution

It presents a new 360V dataset with multi-modal data and weak layout cues, and proposes an explicit coupling method for simultaneous depth and layout estimation from monocular spherical images.

Findings

Enhanced accuracy in depth estimation and layout prediction.

Effective integration of layout cues improves geometric perception.

Facilitates quick building-scale 3D scanning using single 360 cameras.

Abstract

Spherical cameras capture scenes in a holistic manner and have been used for room layout estimation. Recently, with the availability of appropriate datasets, there has also been progress in depth estimation from a single omnidirectional image. While these two tasks are complementary, few works have been able to explore them in parallel to advance indoor geometric perception, and those that have done so either relied on synthetic data, or used small scale datasets, as few options are available that include both layout annotations and dense depth maps in real scenes. This is partly due to the necessity of manual annotations for room layouts. In this work, we move beyond this limitation and generate a 360 geometric vision (360V) dataset that includes multiple modalities, multi-view stereo data and automatically generated weak layout cues. We also explore an explicit coupling between the two tasks to integrate them into a singleshot trained model. We rely on depth-based layout reconstruction and layout-based depth attention, demonstrating increased performance across both tasks. By using single 360 cameras to scan rooms, the opportunity for facile and quick building-scale 3D scanning arises.