Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling

TL;DR

Structured3D is a large-scale, photo-realistic synthetic dataset with detailed 3D structure annotations, designed to improve learning-based 3D scene modeling and understanding methods.

Contribution

The paper introduces Structured3D, a synthetic dataset with rich 3D annotations, generated from professional interior designs, to facilitate research in structured 3D modeling.

Findings

Enhanced room layout estimation performance using the dataset.

Combining synthetic and real images improves deep network training.

Provides a scalable alternative to human-annotated 3D data.

Abstract

Recently, there has been growing interest in developing learning-based methods to detect and utilize salient semi-global or global structures, such as junctions, lines, planes, cuboids, smooth surfaces, and all types of symmetries, for 3D scene modeling and understanding. However, the ground truth annotations are often obtained via human labor, which is particularly challenging and inefficient for such tasks due to the large number of 3D structure instances (e.g., line segments) and other factors such as viewpoints and occlusions. In this paper, we present a new synthetic dataset, Structured3D, with the aim of providing large-scale photo-realistic images with rich 3D structure annotations for a wide spectrum of structured 3D modeling tasks. We take advantage of the availability of professional interior designs and automatically extract 3D structures from them. We generate high-quality images with an industry-leading rendering engine. We use our synthetic dataset in combination with real images to train deep networks for room layout estimation and demonstrate improved performance on benchmark datasets.