NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments

TL;DR

NavMorph introduces a self-evolving world model with contextual memory for improved generalization and adaptability in vision-and-language navigation tasks within complex, continuous environments.

Contribution

It presents a novel self-evolving world model framework with contextual evolution memory, enhancing environmental understanding and online adaptability in VLN-CE tasks.

Findings

Significant performance improvements on VLN-CE benchmarks.

Effective environmental modeling with compact latent representations.

Enhanced online adaptability and decision-making in navigation.

Abstract

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to execute sequential navigation actions in complex environments guided by natural language instructions. Current approaches often struggle with generalizing to novel environments and adapting to ongoing changes during navigation. Inspired by human cognition, we present NavMorph, a self-evolving world model framework that enhances environmental understanding and decision-making in VLN-CE tasks. NavMorph employs compact latent representations to model environmental dynamics, equipping agents with foresight for adaptive planning and policy refinement. By integrating a novel Contextual Evolution Memory, NavMorph leverages scene-contextual information to support effective navigation while maintaining online adaptability. Extensive experiments demonstrate that our method achieves notable performance improvements on popular VLN-CE benchmarks. Code is available at https://github.com/Feliciaxyao/NavMorph.