History-Enhanced Two-Stage Transformer for Aerial Vision-and-Language Navigation

TL;DR

This paper introduces HETT, a two-stage transformer framework that improves aerial navigation by combining global reasoning and local scene understanding through a coarse-to-fine approach and structured spatial memory.

Contribution

The work presents a novel two-stage transformer model with a historical grid map for enhanced scene awareness in UAV navigation, along with refined dataset annotations.

Findings

HETT outperforms existing methods on the CityNav dataset.

The historical grid map improves spatial memory and scene understanding.

Ablation studies confirm the effectiveness of each component.

Abstract

Aerial Vision-and-Language Navigation (AVLN) requires Unmanned Aerial Vehicle (UAV) agents to localize targets in large-scale urban environments based on linguistic instructions. While successful navigation demands both global environmental reasoning and local scene comprehension, existing UAV agents typically adopt mono-granularity frameworks that struggle to balance these two aspects. To address this limitation, this work proposes a History-Enhanced Two-Stage Transformer (HETT) framework, which integrates the two aspects through a coarse-to-fine navigation pipeline. Specifically, HETT first predicts coarse-grained target positions by fusing spatial landmarks and historical context, then refines actions via fine-grained visual analysis. In addition, a historical grid map is designed to dynamically aggregate visual features into a structured spatial memory, enhancing comprehensive scene awareness. Additionally, the CityNav dataset annotations are manually refined to enhance data quality. Experiments on the refined CityNav dataset show that HETT delivers significant performance gains, while extensive ablation studies further verify the effectiveness of each component.