Building Instance Classification Using Street View Images

TL;DR

This paper introduces a CNN-based framework that combines street view images and remote sensing data to classify individual building functions, improving boundary detection and urban mapping accuracy.

Contribution

It presents a novel method integrating street view images with remote sensing data for detailed building classification and creates a benchmark dataset for training and evaluation.

Findings

Effective classification of building functions using street view and remote sensing images.

Generated detailed building maps for multiple cities in Canada and the US.

Demonstrated improved boundary detection for individual buildings.

Abstract

Land-use classification based on spaceborne or aerial remote sensing images has been extensively studied over the past decades. Such classification is usually a patch-wise or pixel-wise labeling over the whole image. But for many applications, such as urban population density mapping or urban utility planning, a classification map based on individual buildings is much more informative. However, such semantic classification still poses some fundamental challenges, for example, how to retrieve fine boundaries of individual buildings. In this paper, we proposed a general framework for classifying the functionality of individual buildings. The proposed method is based on Convolutional Neural Networks (CNNs) which classify facade structures from street view images, such as Google StreetView, in addition to remote sensing images which usually only show roof structures. Geographic information was utilized to mask out individual buildings, and to associate the corresponding street view images. We created a benchmark dataset which was used for training and evaluating CNNs. In addition, the method was applied to generate building classification maps on both region and city scales of several cities in Canada and the US. Keywords: CNN, Building instance classification, Street view images, OpenStreetMap