Room Envelopes: A Synthetic Dataset for Indoor Layout Reconstruction from Images

TL;DR

This paper introduces Room Envelopes, a synthetic dataset designed to improve indoor scene reconstruction by providing RGB images and structural layout pointmaps, enabling better prediction of scene extents and structural elements.

Contribution

The paper presents a novel synthetic dataset, Room Envelopes, with paired RGB images and layout pointmaps to facilitate supervised learning of indoor structural scene elements.

Findings

Enables supervised training of monocular geometry estimators.

Improves prediction of scene extent and structural layout.

Supports better reconstruction of occluded surfaces.

Abstract

Modern scene reconstruction methods are able to accurately recover 3D surfaces that are visible in one or more images. However, this leads to incomplete reconstructions, missing all occluded surfaces. While much progress has been made on reconstructing entire objects given partial observations using generative models, the structural elements of a scene, like the walls, floors and ceilings, have received less attention. We argue that these scene elements should be relatively easy to predict, since they are typically planar, repetitive and simple, and so less costly approaches may be suitable. In this work, we present a synthetic dataset -- Room Envelopes -- that facilitates progress on this task by providing a set of RGB images and two associated pointmaps for each image: one capturing the visible surface and one capturing the first surface once fittings and fixtures are removed, that is, the structural layout. As we show, this enables direct supervision for feed-forward monocular geometry estimators that predict both the first visible surface and the first layout surface. This confers an understanding of the scene's extent, as well as the shape and location of its objects.