Vectorizing World Buildings: Planar Graph Reconstruction by Primitive Detection and Relationship Inference

TL;DR

This paper introduces a new method for converting 2D images of outdoor buildings into detailed planar graphs by detecting geometric primitives and their relationships, supported by a new benchmark dataset.

Contribution

It presents a novel CNN-based primitive detection and relationship inference algorithm combined with integer programming for accurate 2D building graph reconstruction, along with a new annotated dataset.

Findings

Significant improvements over existing methods in graph reconstruction accuracy

Effective primitive detection and relationship inference demonstrated

Benchmark dataset enables future research in building vectorization

Abstract

This paper tackles a 2D architecture vectorization problem, whose task is to infer an outdoor building architecture as a 2D planar graph from a single RGB image. We provide a new benchmark with ground-truth annotations for 2,001 complex buildings across the cities of Atlanta, Paris, and Las Vegas. We also propose a novel algorithm utilizing 1) convolutional neural networks (CNNs) that detects geometric primitives and infers their relationships and 2) an integer programming (IP) that assembles the information into a 2D planar graph. While being a trivial task for human vision, the inference of a graph structure with an arbitrary topology is still an open problem for computer vision. Qualitative and quantitative evaluations demonstrate that our algorithm makes significant improvements over the current state-of-the-art, towards an intelligent system at the level of human perception. We will share code and data.