Focusing on Shadows for Predicting Heightmaps from Single Remotely Sensed RGB Images with Deep Learning

TL;DR

This paper introduces a deep learning approach that leverages shadow maps in single aerial images to accurately predict heightmaps, aiding 3D understanding in remote sensing without extensive additional data.

Contribution

The novel contribution is the integration of shadow information into a deep learning model for heightmap prediction from single RGB images, improving accuracy and efficiency.

Findings

The model outperforms existing methods on Manchester and IEEE datasets.

Shadow information significantly enhances heightmap estimation accuracy.

Efficient shadow computation adds minimal extra complexity.

Abstract

Estimating the heightmaps of buildings and vegetation in single remotely sensed images is a challenging problem. Effective solutions to this problem can comprise the stepping stone for solving complex and demanding problems that require 3D information of aerial imagery in the remote sensing discipline, which might be expensive or not feasible to require. We propose a task-focused Deep Learning (DL) model that takes advantage of the shadow map of a remotely sensed image to calculate its heightmap. The shadow is computed efficiently and does not add significant computation complexity. The model is trained with aerial images and their Lidar measurements, achieving superior performance on the task. We validate the model with a dataset covering a large area of Manchester, UK, as well as the 2018 IEEE GRSS Data Fusion Contest Lidar dataset. Our work suggests that the proposed DL architecture and the technique of injecting shadows information into the model are valuable for improving the heightmap estimation task for single remotely sensed imagery.