Enhancing Monocular Height Estimation via Sparse LiDAR-Guided Correction

TL;DR

This paper presents a novel automated pipeline that combines sparse global LiDAR data with deep learning to significantly improve monocular height estimation accuracy across diverse environments, enabling scalable 3D mapping.

Contribution

It introduces a fully automated correction method integrating LiDAR and deep learning, establishing the first benchmark for this approach and demonstrating substantial accuracy improvements.

Findings

Reduces MHE MAE by 30.9%

Improves F1HE score by 44.2% for MHE

Enhances MDE MAE by 24.1%

Abstract

Monocular height estimation (MHE) from very-high-resolution (VHR) optical imagery remains challenging due to limited structural cues and the high cost and geographic constraints of conventional elevation data such as airborne LiDAR and multi-view stereo. Although recent MHE and monocular depth estimation (MDE) models show strong performance, their robustness under varied illumination and scene conditions is still limited. We introduce a fully automated correction pipeline that integrates sparse, imperfect global LiDAR measurements from ICESat-2 with deep learning predictions to enhance accuracy and stability. The workflow relies entirely on publicly available models and data and requires only a single georeferenced optical image to produce corrected height maps, enabling low-cost and globally scalable deployment. We also establish the first benchmark for this task, evaluating two random forest based approaches, four parameter efficient fine tuning methods, and full fine tuning. Experiments across six diverse regions at 0.5 m resolution (297 km2), covering the urban cores of Tokyo, Paris, and Sao Paulo as well as suburban and forested areas, show substantial gains. The best method reduces the MHE model's mean absolute error (MAE) by 30.9 percent and improves its F1HE score by 44.2 percent. For the MDE model, MAE improves by 24.1 percent and the F1HE score by 25.1 percent. These results validate the effectiveness of our correction pipeline and demonstrate how sparse global LiDAR can systematically strengthen both MHE and MDE models, enabling scalable and widely accessible 3D height mapping.