Single-Shot Cuboids: Geodesics-based End-to-end Manhattan Aligned Layout Estimation from Spherical Panoramas

TL;DR

This paper introduces a novel end-to-end method for estimating Manhattan-aligned room layouts from spherical panoramas without intermediate steps, using direct coordinate regression and geodesic-aware loss functions.

Contribution

It is the first to directly infer Manhattan-aligned layouts in a single shot, removing the need for postprocessing and intermediate representations.

Findings

Achieves accurate Manhattan-aligned layout estimation from spherical panoramas.

Introduces geodesic heatmaps and loss for better keypoint detection.

Provides publicly available models and code for the community.

Abstract

It has been shown that global scene understanding tasks like layout estimation can benefit from wider field of views, and specifically spherical panoramas. While much progress has been made recently, all previous approaches rely on intermediate representations and postprocessing to produce Manhattan-aligned estimates. In this work we show how to estimate full room layouts in a single-shot, eliminating the need for postprocessing. Our work is the first to directly infer Manhattan-aligned outputs. To achieve this, our data-driven model exploits direct coordinate regression and is supervised end-to-end. As a result, we can explicitly add quasi-Manhattan constraints, which set the necessary conditions for a homography-based Manhattan alignment module. Finally, we introduce the geodesic heatmaps and loss and a boundary-aware center of mass calculation that facilitate higher quality keypoint estimation in the spherical domain. Our models and code are publicly available at https://vcl3d.github.io/SingleShotCuboids/.