DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama
Shang-Ta Yang, Fu-En Wang, Chi-Han Peng, Peter Wonka, Min Sun,, Hung-Kuo Chu

TL;DR
DuLa-Net is a deep learning framework that predicts 3D room layouts from a single RGB panorama by leveraging dual projections and a novel feature fusion, outperforming previous methods especially on complex room shapes.
Contribution
Introduces DuLa-Net, a dual-projection neural network with a new feature fusion method for improved room layout estimation from panoramas.
Findings
Outperforms state-of-the-art in accuracy and performance.
Effective on rooms with non-cuboid layouts.
Introduces the Realtor360 dataset for complex layouts.
Abstract
We present a deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama. To achieve better prediction accuracy, our method leverages two projections of the panorama at once, namely the equirectangular panorama-view and the perspective ceiling-view, that each contains different clues about the room layouts. Our network architecture consists of two encoder-decoder branches for analyzing each of the two views. In addition, a novel feature fusion structure is proposed to connect the two branches, which are then jointly trained to predict the 2D floor plans and layout heights. To learn more complex room layouts, we introduce the Realtor360 dataset that contains panoramas of Manhattan-world room layouts with different numbers of corners. Experimental results show that our work outperforms recent state-of-the-art in prediction accuracy and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · 3D Surveying and Cultural Heritage · Video Surveillance and Tracking Methods
