SEDualVLN: A Spatially-Enhanced Dual-System for Vision-Language Navigation

TL;DR

SEDualVLN introduces a dual-system framework combining spatially-aware vision-language models and large language models with mapping for improved vision-language navigation, achieving state-of-the-art results.

Contribution

The paper presents SEDualVLN, a novel dual-system approach that enhances spatial reasoning and cooperation between models for better navigation performance.

Findings

Achieves state-of-the-art results on VLN-CE benchmarks.

Demonstrates effectiveness of spatial enhancements in both systems.

Shows improved long-horizon navigation and planning capabilities.

Abstract

Vision-Language Navigation (VLN) approaches have currently followed two primary paradigms: the end-to-end Vision-Language Model (VLM) policy fine-tuned on navigation trajectories to directly predict actions, and the zero-shot modular pipeline integrating pre-trained Multimodal Large Language Model (MLLM) for training-free generalization to unseen environments. However, end-to-end methods struggle with long-horizon navigation and lack dynamic reasoning, whereas zero-shot methods are constrained by limited spatial grounding for reliable planning and also require substantial reasoning time. To bridge this gap, we introduce SEDualVLN, a spatially-enhanced dual-system VLN framework. System 1 is a VLM model enhanced with both global and local spatial awareness, used for action generation. System 2 integrates a general MLLM with a mapping module, wherein the MLLM plans waypoints by leveraging top-down views of the real-time 3D map alongside streams of rendered path images. Both systems leverage different forms of spatial enhancement to cultivate the agent's sense of direction in VLN tasks. Ultimately, they cooperate to complete the navigation task through a fast-slow coordinated approach. SEDualVLN achieves state-of-the-art performance on VLN-CE benchmarks, and further ablation studies demonstrate the effectiveness of each system and module.