LookasideVLN: Direction-Aware Aerial Vision-and-Language Navigation

TL;DR

LookasideVLN introduces a novel aerial navigation approach that leverages directional cues in natural language to improve spatial reasoning and computational efficiency in UAV navigation tasks.

Contribution

It proposes a new paradigm with three core components that utilize directional cues for better navigation accuracy and efficiency in complex urban environments.

Findings

Outperforms state-of-the-art CityNavAgent in navigation tasks.

Uses directional cues to enhance spatial reasoning.

Achieves significant improvements with a single-level lookahead.

Abstract

Aerial Vision-and-Language Navigation (Aerial VLN) enables unmanned aerial vehicles (UAVs) to follow natural language instructions and navigate complex urban environments. While recent advances have achieved progress through large-scale memory graphs and lookahead path planning, they remain limited by shallow instruction understanding and high computational cost. In particular, existing methods rely primarily on landmark descriptions, overlooking directional cues "a key source of spatial context in human navigation". In this work, we propose LookasideVLN, a new paradigm that exploits directional cues in natural language to achieve both more accurate spatial reasoning and greater computational efficiency. LookasideVLN comprises three core components: (1) an Egocentric Lookaside Graph (ELG) that dynamically encodes instruction-relevant landmarks and their directional relationships, (2) a Spatial Landmark Knowledge Base (SLKB) that provides lightweight memory retrieval from prior navigation experiences, and (3) a Lookaside MLLM Navigation Agent that aligns multimodal information from user instructions, visual observations, and landmark-direction information from ELG for path planning. Extensive experiments show that LookasideVLN significantly outperforms the state-of-the-art CityNavAgent, even with a single-level lookahead, demonstrating that leveraging directional cues is a powerful yet efficient strategy for Aerial VLN.