Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments

TL;DR

This paper introduces CA-Nav, a novel constraint-aware approach for zero-shot vision-language navigation in continuous environments, enabling robots to navigate effectively without prior demonstrations by decomposing instructions and dynamically planning paths.

Contribution

The paper proposes a new constraint-aware framework with modules for sub-instruction management and value mapping, advancing zero-shot VLN-CE performance and stability.

Findings

Achieves 12-13% higher success rates on benchmarks.

Outperforms previous methods in real-world robot tests.

Demonstrates effective navigation without prior training data.

Abstract

We address the task of Vision-Language Navigation in Continuous Environments (VLN-CE) under the zero-shot setting. Zero-shot VLN-CE is particularly challenging due to the absence of expert demonstrations for training and minimal environment structural prior to guide navigation. To confront these challenges, we propose a Constraint-Aware Navigator (CA-Nav), which reframes zero-shot VLN-CE as a sequential, constraint-aware sub-instruction completion process. CA-Nav continuously translates sub-instructions into navigation plans using two core modules: the Constraint-Aware Sub-instruction Manager (CSM) and the Constraint-Aware Value Mapper (CVM). CSM defines the completion criteria for decomposed sub-instructions as constraints and tracks navigation progress by switching sub-instructions in a constraint-aware manner. CVM, guided by CSM's constraints, generates a value map on the fly and refines it using superpixel clustering to improve navigation stability. CA-Nav achieves the state-of-the-art performance on two VLN-CE benchmarks, surpassing the previous best method by 12 percent and 13 percent in Success Rate on the validation unseen splits of R2R-CE and RxR-CE, respectively. Moreover, CA-Nav demonstrates its effectiveness in real-world robot deployments across various indoor scenes and instructions.