Towards Open Environments and Instructions: General Vision-Language Navigation via Fast-Slow Interactive Reasoning

TL;DR

This paper introduces a dynamic fast-slow reasoning framework for vision-language navigation, enabling agents to adapt to diverse and unseen environments by enhancing generalization through interactive cognition modules.

Contribution

It proposes a novel fast-slow interactive reasoning framework that improves generalization in open-world VLN tasks by continuously optimizing fast decision-making with slow reflection.

Findings

Outperforms existing methods in generalization to unseen environments

Enhances navigation accuracy through fast-slow reasoning interaction

Demonstrates robustness in diverse and inconsistent instructions

Abstract

Vision-Language Navigation (VLN) aims to enable agents to navigate to a target location based on language instructions. Traditional VLN often follows a close-set assumption, i.e., training and test data share the same style of the input images and instructions. However, the real world is open and filled with various unseen environments, posing enormous difficulties for close-set methods. To this end, we focus on the General Scene Adaptation (GSA-VLN) task, aiming to learn generalized navigation ability by introducing diverse environments and inconsistent instructions.Recent research indicates that by means of fast and slow cognition systems, human beings could generate stable policies, which strengthen their adaptation for open world. Inspired by this idea, we propose the slow4fast-VLN, establishing a dynamic interactive fast-slow reasoning framework. The fast-reasoning module, an end-to-end strategy network, outputs actions via real-time input. It accumulates execution records in a history repository to build memory. The slow-reasoning module analyze the memories generated by the fast-reasoning module. Through deep reflection, it extracts experiences that enhance the generalization ability of decision-making. These experiences are structurally stored and used to continuously optimize the fast-reasoning module. Unlike traditional methods that treat fast-slow reasoning as independent mechanisms, our framework enables fast-slow interaction. By leveraging the experiences from slow reasoning, it continually improves the accuracy and generalization ability of fast decisions. This interaction allows the system to continuously adapt and efficiently execute navigation tasks when facing unseen scenarios. Extensive experiments demonstrate the superiorities of our method.