Combining Maps and Street Level Images for Building Height and Facade Estimation

TL;DR

This paper introduces a method that combines 2D building footprints and street-level images to accurately estimate building heights and facades, overcoming challenges like camera position errors.

Contribution

The method integrates map data and images to estimate building heights and facades, including a novel camera localization approach to improve accuracy.

Findings

Effective in estimating building heights from street images.

Improves camera position accuracy using map-image correspondences.

Demonstrates promising results on real-world datasets.

Abstract

We propose a method that integrates two widely available data sources, building footprints from 2D maps and street level images, to derive valuable information that is generally difficult to acquire -- building heights and building facade masks in images. Building footprints are elevated in world coordinates and projected onto images. Building heights are estimated by scoring projected footprints based on their alignment with building features in images. Building footprints with estimated heights can be converted to simple 3D building models, which are projected back to images to identify buildings. In this procedure, accurate camera projections are critical. However, camera position errors inherited from external sensors commonly exist, which adversely affect results. We derive a solution to precisely locate cameras on maps using correspondence between image features and building footprints. Experiments on real-world datasets show the promise of our method.