Holistic Multi-View Building Analysis in the Wild with Projection Pooling

TL;DR

This paper introduces a new large-scale dataset and a novel projection pooling layer for multi-view building classification, improving accuracy in analyzing fine-grained building attributes from diverse urban images.

Contribution

The paper presents a new benchmarking dataset of urban building images with metadata and a projection pooling layer that enhances multi-view building analysis accuracy.

Findings

Projection pooling improves classification accuracy.

The dataset captures real-world urban building challenges.

The method effectively combines top-view and street-view imagery.

Abstract

We address six different classification tasks related to fine-grained building attributes: construction type, number of floors, pitch and geometry of the roof, facade material, and occupancy class. Tackling such a remote building analysis problem became possible only recently due to growing large-scale datasets of urban scenes. To this end, we introduce a new benchmarking dataset, consisting of 49426 images (top-view and street-view) of 9674 buildings. These photos are further assembled, together with the geometric metadata. The dataset showcases various real-world challenges, such as occlusions, blur, partially visible objects, and a broad spectrum of buildings. We propose a new projection pooling layer, creating a unified, top-view representation of the top-view and the side views in a high-dimensional space. It allows us to utilize the building and imagery metadata seamlessly. Introducing this layer improves classification accuracy -- compared to highly tuned baseline models -- indicating its suitability for building analysis.