LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment

TL;DR

LoD-Loc introduces a UAV visual localization method using LoD 3D maps and neural wireframe alignment, achieving high accuracy without complex 3D models.

Contribution

It presents a novel localization approach that aligns wireframes from LoD maps with neural predictions, outperforming textured 3D model-based methods.

Findings

Achieves superior localization accuracy compared to state-of-the-art methods.

Creates new LoD3.0 and LoD2.0 datasets for UAV localization.

Demonstrates effective pose refinement with a differentiable Gaussian-Newton algorithm.

Abstract

We propose a new method named LoD-Loc for visual localization in the air. Unlike existing localization algorithms, LoD-Loc does not rely on complex 3D representations and can estimate the pose of an Unmanned Aerial Vehicle (UAV) using a Level-of-Detail (LoD) 3D map. LoD-Loc mainly achieves this goal by aligning the wireframe derived from the LoD projected model with that predicted by the neural network. Specifically, given a coarse pose provided by the UAV sensor, LoD-Loc hierarchically builds a cost volume for uniformly sampled pose hypotheses to describe pose probability distribution and select a pose with maximum probability. Each cost within this volume measures the degree of line alignment between projected and predicted wireframes. LoD-Loc also devises a 6-DoF pose optimization algorithm to refine the previous result with a differentiable Gaussian-Newton method. As no public dataset exists for the studied problem, we collect two datasets with map levels of LoD3.0 and LoD2.0, along with real RGB queries and ground-truth pose annotations. We benchmark our method and demonstrate that LoD-Loc achieves excellent performance, even surpassing current state-of-the-art methods that use textured 3D models for localization. The code and dataset are available at https://victorzoo.github.io/LoD-Loc.github.io/.