Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction

TL;DR

This paper introduces Conv-MPN, a convolutional message passing neural network designed to reconstruct outdoor building structures as planar graphs from single RGB images, improving accuracy over previous neural methods.

Contribution

The paper presents a novel Conv-MPN architecture that uses feature volumes and convolutional message passing for structured outdoor architecture reconstruction from images.

Findings

Significant performance improvements over existing neural solutions.

Effective reconstruction of building planar graphs from single RGB images.

Potential to inspire new research in graph neural networks for geometry reconstruction.

Abstract

This paper proposes a novel message passing neural (MPN) architecture Conv-MPN, which reconstructs an outdoor building as a planar graph from a single RGB image. Conv-MPN is specifically designed for cases where nodes of a graph have explicit spatial embedding. In our problem, nodes correspond to building edges in an image. Conv-MPN is different from MPN in that 1) the feature associated with a node is represented as a feature volume instead of a 1D vector; and 2) convolutions encode messages instead of fully connected layers. Conv-MPN learns to select a true subset of nodes (i.e., building edges) to reconstruct a building planar graph. Our qualitative and quantitative evaluations over 2,000 buildings show that Conv-MPN makes significant improvements over the existing fully neural solutions. We believe that the paper has a potential to open a new line of graph neural network research for structured geometry reconstruction.