SeqWalker: Sequential-Horizon Vision-and-Language Navigation with Hierarchical Planning

TL;DR

SeqWalker is a hierarchical navigation model designed for complex, multi-task vision-and-language instructions, improving agent focus and accuracy in long-horizon tasks through dynamic planning and structured error correction.

Contribution

The paper introduces SeqWalker, a hierarchical planning framework with high-level dynamic instruction selection and low-level exploration-verification, addressing information overload in multi-task navigation.

Findings

SeqWalker outperforms existing models on extended IVLN benchmark.

Hierarchical planning reduces cognitive load and improves navigation accuracy.

Structured error correction enhances trajectory reliability.

Abstract

Sequential-Horizon Vision-and-Language Navigation (SH-VLN) presents a challenging scenario where agents should sequentially execute multi-task navigation guided by complex, long-horizon language instructions. Current vision-and-language navigation models exhibit significant performance degradation with such multi-task instructions, as information overload impairs the agent's ability to attend to observationally relevant details. To address this problem, we propose SeqWalker, a navigation model built on a hierarchical planning framework. Our SeqWalker features: i) A High-Level Planner that dynamically selects global instructions into contextually relevant sub-instructions based on the agent's current visual observations, thus reducing cognitive load; ii) A Low-Level Planner incorporating an Exploration-Verification strategy that leverages the inherent logical structure of instructions for trajectory error correction. To evaluate SH-VLN performance, we also extend the IVLN dataset and establish a new benchmark. Extensive experiments are performed to demonstrate the superiority of the proposed SeqWalker.