# A Multi-Modal, Multi-Temporal, Multi-Resolution Benchmark Dataset for Building Height Estimation

## TL;DR

This paper introduces a large satellite-based dataset for estimating building heights, enabling scalable and frequent urban monitoring.

## Contribution

The novel contribution is M4Heights, a multi-modal, multi-temporal, and multi-resolution dataset for building height estimation.

## Key findings

- M4Heights includes ≈1 million images with diverse satellite data and reference height maps.
- The dataset supports deep learning models and includes a super-resolution component for improved accuracy.
- It enables scalable building height estimation across different urban and geographic conditions.

## Abstract

Building heights are crucial for sustainable urban planning and monitoring. While traditional methods use airborne stereo images and LiDAR data for accurate height estimation, their large-scale application is costly and slow, limiting the ability to conduct frequent large-scale monitoring. In contrast, satellite data offers a scalable alternative, further improved with Deep Learning (DL) models. However, the lack of representative open-source training datasets has constrained the progress in this field. In this paper, we introduce M4Heights, a multi-modal, multi-resolution, and multi-temporal dataset designed for building height estimation, spanning diverse architectural styles, urban densities, and terrain complexities across Estonia, Netherlands, and Switzerland. The dataset includes  ≈ 1 million images, comprising time series of Sentinel-1 SAR and Sentinel-2 MSI satellite data, high-resolution aerial orthophotos, and high-quality building height reference maps. Additionally, M4Heights provides the largest associated multi-image super-resolution dataset to enhance height estimation accuracy. Our dataset supports a range of modeling approaches, offers extensibility to new geographic regions and provides opportunities to advance the development of DL models for building height estimation.