A Unified Model for Near and Remote Sensing

TL;DR

This paper introduces a unified neural network model that combines overhead and ground-level images to improve the accuracy of geospatial function estimation, such as land use and building age, through end-to-end training and density estimation.

Contribution

The paper presents a novel neural network architecture that integrates overhead and ground-level imagery for enhanced geospatial analysis, a first in combining these data sources end-to-end.

Findings

More accurate land use classification

Improved building function estimation

Significant accuracy gains in geospatial tasks

Abstract

We propose a novel convolutional neural network architecture for estimating geospatial functions such as population density, land cover, or land use. In our approach, we combine overhead and ground-level images in an end-to-end trainable neural network, which uses kernel regression and density estimation to convert features extracted from the ground-level images into a dense feature map. The output of this network is a dense estimate of the geospatial function in the form of a pixel-level labeling of the overhead image. To evaluate our approach, we created a large dataset of overhead and ground-level images from a major urban area with three sets of labels: land use, building function, and building age. We find that our approach is more accurate for all tasks, in some cases dramatically so.