Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

TL;DR

This paper introduces a divide-and-conquer approach for indoor panorama 3D reconstruction that leverages scene geometry, a yaw-invariant reparameterization for CNNs, and a new benchmark dataset, significantly improving plane detection accuracy.

Contribution

It proposes a novel divide-and-conquer strategy and yaw-invariant CNN reparameterization for better indoor panorama planar reconstruction, along with a new benchmark dataset.

Findings

Our method outperforms state-of-the-art baselines on the PanoH&V dataset.

The yaw-invariant reparameterization improves vertical plane detection accuracy.

The divide-and-conquer approach simplifies plane clustering in panoramic images.

Abstract

Indoor panorama typically consists of human-made structures parallel or perpendicular to gravity. We leverage this phenomenon to approximate the scene in a 360-degree image with (H)orizontal-planes and (V)ertical-planes. To this end, we propose an effective divide-and-conquer strategy that divides pixels based on their plane orientation estimation; then, the succeeding instance segmentation module conquers the task of planes clustering more easily in each plane orientation group. Besides, parameters of V-planes depend on camera yaw rotation, but translation-invariant CNNs are less aware of the yaw change. We thus propose a yaw-invariant V-planar reparameterization for CNNs to learn. We create a benchmark for indoor panorama planar reconstruction by extending existing 360 depth datasets with ground truth H\&V-planes (referred to as PanoH&V dataset) and adopt state-of-the-art planar reconstruction methods to predict H\&V-planes as our baselines. Our method outperforms the baselines by a large margin on the proposed dataset.