PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment

TL;DR

PixCuboid is an optimization-based multi-view method for estimating cuboid-shaped room layouts, outperforming existing single-view approaches and adaptable to multi-room scenarios, validated on new benchmarks.

Contribution

Introduces PixCuboid, a novel multi-view dense feature alignment approach for room layout estimation, with end-to-end training and new benchmarks for evaluation.

Findings

Significantly outperforms existing methods on new benchmarks.

Learned feature maps enable robust multi-view alignment.

Flexible extension to multi-room layouts demonstrated.

Abstract

Coarse room layout estimation provides important geometric cues for many downstream tasks. Current state-of-the-art methods are predominantly based on single views and often assume panoramic images. We introduce PixCuboid, an optimization-based approach for cuboid-shaped room layout estimation, which is based on multi-view alignment of dense deep features. By training with the optimization end-to-end, we learn feature maps that yield large convergence basins and smooth loss landscapes in the alignment. This allows us to initialize the room layout using simple heuristics. For the evaluation we propose two new benchmarks based on ScanNet++ and 2D-3D-Semantics, with manually verified ground truth 3D cuboids. In thorough experiments we validate our approach and significantly outperform the competition. Finally, while our network is trained with single cuboids, the flexibility of the optimization-based approach allow us to easily extend to multi-room estimation, e.g. larger apartments or offices. Code and model weights are available at https://github.com/ghanning/PixCuboid.