Let's Reward Step-by-Step: Step-Aware Contrastive Alignment for Vision-Language Navigation in Continuous Environments

TL;DR

This paper introduces Step-Aware Contrastive Alignment (SACA), a novel framework that improves vision-language navigation in continuous environments by providing dense, step-by-step supervision to enhance learning, generalization, and error recovery.

Contribution

SACA is the first method to extract dense supervision from imperfect trajectories using a perception-grounded auditor and dynamic batch routing, significantly advancing VLN-CE performance.

Findings

SACA achieves state-of-the-art results on VLN-CE benchmarks.

The step-aware approach improves error recovery and generalization.

Dense supervision from trajectories enhances training stability.

Abstract

Vision-Language Navigation in Continuous Environments (VLN-CE) requires agents to learn complex reasoning from long-horizon human interactions. While Multi-modal Large Language Models (MLLMs) have driven recent progress, current training paradigms struggle to balance generalization capability, error recovery and training stability. Specifically, (i) policies derived from SFT suffer from compounding errors, struggling to recover from out-of-distribution states, and (ii) Reinforcement Fine-Tuning (RFT) methods e.g. GRPO are bottlenecked by sparse outcome rewards. Their binary feedback fails to assign credit to individual steps, leading to gradient signal collapse in failure dominant batches. To address these challenges, we introduce Step-Aware Contrastive Alignment (SACA), a framework designed to extract dense supervision from imperfect trajectories. At its core, the Perception-Grounded Step-Aware auditor evaluates progress step-by-step, disentangling failed trajectories into valid prefixes and exact divergence points. Leveraging these signals, Scenario-Conditioned Group Construction mechanism dynamically routes batches to specialized resampling and optimization strategies. Extensive experiments on VLN-CE benchmarks demonstrate that SACA achieves state-of-the-art performance.