LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image

TL;DR

This paper introduces LayoutNet, an algorithm that reconstructs 3D room layouts from a single RGB image, effectively handling panoramas, perspective images, cuboid, and complex layouts with improved accuracy and speed.

Contribution

LayoutNet is the first method to operate directly on panoramic images for 3D room layout reconstruction, improving accuracy and generalization over existing approaches.

Findings

Performs well on panoramic images in speed and accuracy.

Achieves high accuracy on perspective images.

Handles both cuboid and complex Manhattan layouts.

Abstract

We propose an algorithm to predict room layout from a single image that generalizes across panoramas and perspective images, cuboid layouts and more general layouts (e.g. L-shape room). Our method operates directly on the panoramic image, rather than decomposing into perspective images as do recent works. Our network architecture is similar to that of RoomNet, but we show improvements due to aligning the image based on vanishing points, predicting multiple layout elements (corners, boundaries, size and translation), and fitting a constrained Manhattan layout to the resulting predictions. Our method compares well in speed and accuracy to other existing work on panoramas, achieves among the best accuracy for perspective images, and can handle both cuboid-shaped and more general Manhattan layouts.