NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance

TL;DR

NavDP introduces a transformer-based, end-to-end navigation policy trained in simulation that achieves zero-shot transfer to real-world environments, outperforming previous methods and leveraging privileged simulation data for improved spatial understanding.

Contribution

The paper presents NavDP, a novel diffusion-based navigation policy that learns from simulation with privileged information, enabling effective zero-shot sim-to-real transfer for autonomous robot navigation.

Findings

NavDP outperforms prior state-of-the-art methods in real-world tests.

The approach effectively leverages privileged simulation data for spatial understanding.

Key factors influencing generalization performance are identified.

Abstract

Learning to navigate in dynamic and complex open-world environments is a critical yet challenging capability for autonomous robots. Existing approaches often rely on cascaded modular frameworks, which require extensive hyperparameter tuning or learning from limited real-world demonstration data. In this paper, we propose Navigation Diffusion Policy (NavDP), an end-to-end network trained solely in simulation that enables zero-shot sim-to-real transfer across diverse environments and robot embodiments. The core of NavDP is a unified transformer-based architecture that jointly learns trajectory generation and trajectory evaluation, both conditioned solely on local RGB-D observation. By learning to predict critic values for contrastive trajectory samples, our proposed approach effectively leverages supervision from privileged information available in simulation, thereby fostering accurate spatial understanding and enabling the distinction between safe and dangerous behaviors. To support this, we develop an efficient data generation pipeline in simulation and construct a large-scale dataset encompassing over one million meters of navigation experience across 3,000 scenes. Empirical experiments in both simulated and real-world environments demonstrate that NavDP significantly outperforms prior state-of-the-art methods. Furthermore, we identify key factors influencing the generalization performance of NavDP. The dataset and code are publicly available at https://wzcai99.github.io/navigation-diffusion-policy.github.io.