Lights Out: A Nighttime UAV Localization Framework Using Thermal Imagery and Semantic 3D Maps

TL;DR

This paper introduces a semantic reprojection framework for nighttime UAV localization using thermal imagery and semantic 3D maps, achieving sub-2 meter accuracy without GNSS.

Contribution

It presents a novel semantic reprojection approach that aligns thermal observations with semantic 3D maps for robust nighttime UAV localization.

Findings

Achieves a median RMSE of 1.52 meters in real-world nighttime UAV flights.

Localization accuracy improves with the availability of semantic edge evidence.

Large localization errors are mainly in semantically ambiguous areas.

Abstract

Reliable backup localization for unmanned aerial vehicles (UAVs) operating in GNSS-denied nighttime conditions remains an open challenge due to the severe modality gap between daytime RGB maps and nighttime thermal imagery. This work presents a semantic reprojection framework for map-relative nighttime UAV localization by aligning segmented thermal observations with a globally referenced, semantically labeled 3D map constructed from daytime RGB data. Rather than relying on appearance-based correspondence, localization is formulated in a shared semantic domain and solved via a symmetric bidirectional reprojection objective with confusion-aware weighting to improve robustness under segmentation uncertainty. The approach is evaluated offline across 6.5 km of nighttime, real-world UAV flight trajectories in urban and semi-structured environments. Relative to RTK GNSS ground truth, the system achieves a bias-corrected RMSE2D of 2.18 m and a median RMSE2D of 1.52 m. Results show that localization performance is strongly correlated with the availability of semantic edge evidence and that large-error events are spatially localized to semantically ambiguous areas rather than uniformly distributed. These findings indicate that semantic reprojection offers a promising pathway toward globally referenced nighttime UAV localization using thermal imagery alone.