CityNav: A Large-Scale Dataset for Real-World Aerial Navigation

TL;DR

CityNav introduces a large-scale real-world dataset for aerial vision-and-language navigation, enabling research on agents interpreting geographic and visual cues in city environments.

Contribution

We present the first large-scale real-world aerial VLN dataset, CityNav, and a methodology for creating geographic semantic maps to enhance navigation performance.

Findings

Semantic maps significantly improve agent navigation accuracy.

AerialVLN models outperform baseline methods on CityNav.

CityNav covers 4.65 km² in Cambridge and Birmingham with 32,637 trajectories.

Abstract

Vision-and-language navigation (VLN) aims to develop agents capable of navigating in realistic environments. While recent cross-modal training approaches have significantly improved navigation performance in both indoor and outdoor scenarios, aerial navigation over real-world cities remains underexplored primarily due to limited datasets and the difficulty of integrating visual and geographic information. To fill this gap, we introduce CityNav, the first large-scale real-world dataset for aerial VLN. Our dataset consists of 32,637 human demonstration trajectories, each paired with a natural language description, covering 4.65 km$^2$ across two real cities: Cambridge and Birmingham. In contrast to existing datasets composed of synthetic scenes such as AerialVLN, our dataset presents a unique challenge because agents must interpret spatial relationships between real-world landmarks and the navigation destination, making CityNav an essential benchmark for advancing aerial VLN. Furthermore, as an initial step toward addressing this challenge, we provide a methodology of creating geographic semantic maps that can be used as an auxiliary modality input during navigation. In our experiments, we compare performance of three representative aerial VLN agents (Seq2seq, CMA and AerialVLN models) and demonstrate that the semantic map representation significantly improves their navigation performance.