OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas

TL;DR

OmniLayout introduces spherical convolutions for room layout reconstruction from panoramic images, significantly improving accuracy by addressing distortion issues inherent in traditional methods.

Contribution

The paper presents a novel spherical convolutional network, OmniLayout, that directly processes panoramic images for improved 3D room layout estimation, outperforming existing methods.

Findings

Reduces error in distorted regions by ~25%.

Outperforms state-of-the-art by ~4% on benchmark datasets.

Uses spherical convolutions for distortion-invariant processing.

Abstract

Given a single RGB panorama, the goal of 3D layout reconstruction is to estimate the room layout by predicting the corners, floor boundary, and ceiling boundary. A common approach has been to use standard convolutional networks to predict the corners and boundaries, followed by post-processing to generate the 3D layout. However, the space-varying distortions in panoramic images are not compatible with the translational equivariance property of standard convolutions, thus degrading performance. Instead, we propose to use spherical convolutions. The resulting network, which we call OmniLayout performs convolutions directly on the sphere surface, sampling according to inverse equirectangular projection and hence invariant to equirectangular distortions. Using a new evaluation metric, we show that our network reduces the error in the heavily distorted regions (near the poles) by approx 25 % when compared to standard convolutional networks. Experimental results show that OmniLayout outperforms the state-of-the-art by approx 4% on two different benchmark datasets (PanoContext and Stanford 2D-3D). Code is available at https://github.com/rshivansh/OmniLayout.