AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings

TL;DR

This paper introduces a novel method for estimating planar surfaces in 3D scenes from posed images, combining multi-view consistent embeddings with geometric clustering to improve accuracy in plane detection.

Contribution

It proposes a new approach that integrates plane embeddings with geometric methods, enhancing plane estimation accuracy and semantic understanding in 3D scenes.

Findings

Our method outperforms existing approaches on ScanNetV2.

Combining embeddings with geometry improves plane clustering accuracy.

Purely geometric methods lack semantic understanding of planes.

Abstract

Extracting planes from a 3D scene is useful for downstream tasks in robotics and augmented reality. In this paper we tackle the problem of estimating the planar surfaces in a scene from posed images. Our first finding is that a surprisingly competitive baseline results from combining popular clustering algorithms with recent improvements in 3D geometry estimation. However, such purely geometric methods are understandably oblivious to plane semantics, which are crucial to discerning distinct planes. To overcome this limitation, we propose a method that predicts multi-view consistent plane embeddings that complement geometry when clustering points into planes. We show through extensive evaluation on the ScanNetV2 dataset that our new method outperforms existing approaches and our strong geometric baseline for the task of plane estimation.