Fine-Grained Land Use Classification at the City Scale Using Ground-Level Images

TL;DR

This paper introduces a novel framework using ground-level Flickr images and a dual-stream CNN to perform fine-grained land use classification across San Francisco, overcoming challenges of data noisiness and lack of overhead imagery.

Contribution

The study presents a new deep learning approach with strategies for noise reduction to classify 45 land use types from user-generated images at city scale.

Findings

Achieved over 29% recall at land parcel level

Developed a dual-stream CNN for object and scene recognition

Demonstrated effective geo-visualization and analysis

Abstract

We perform fine-grained land use mapping at the city scale using ground-level images. Mapping land use is considerably more difficult than mapping land cover and is generally not possible using overhead imagery as it requires close-up views and seeing inside buildings. We postulate that the growing collections of georeferenced, ground-level images suggest an alternate approach to this geographic knowledge discovery problem. We develop a general framework that uses Flickr images to map 45 different land-use classes for the City of San Francisco. Individual images are classified using a novel convolutional neural network containing two streams, one for recognizing objects and another for recognizing scenes. This network is trained in an end-to-end manner directly on the labeled training images. We propose several strategies to overcome the noisiness of our user-generated data including search-based training set augmentation and online adaptive training. We derive a ground truth map of San Francisco in order to evaluate our method. We demonstrate the effectiveness of our approach through geo-visualization and quantitative analysis. Our framework achieves over 29% recall at the individual land parcel level which represents a strong baseline for the challenging 45-way land use classification problem especially given the noisiness of the image data.