Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving

TL;DR

This paper introduces the SNG framework and SNG-VLA model, enhancing end-to-end autonomous driving by effectively integrating global navigation with local scene understanding, leading to improved navigation-following performance.

Contribution

The paper proposes the Sequential Navigation Guidance (SNG) framework and an efficient SNG-VLA model, advancing global and local planning integration in autonomous driving systems.

Findings

SNG-VLA achieves state-of-the-art navigation-following accuracy.

The SNG framework effectively models global navigation information.

SNG-QA dataset aligns global and local planning for autonomous driving.

Abstract

Global navigation information and local scene understanding are two crucial components of autonomous driving systems. However, our experimental results indicate that many end-to-end autonomous driving systems tend to over-rely on local scene understanding while failing to utilize global navigation information. These systems exhibit weak correlation between their planning capabilities and navigation input, and struggle to perform navigation-following in complex scenarios. To overcome this limitation, we propose the Sequential Navigation Guidance (SNG) framework, an efficient representation of global navigation information based on real-world navigation patterns. The SNG encompasses both navigation paths for constraining long-term trajectories and turn-by-turn (TBT) information for real-time decision-making logic. We constructed the SNG-QA dataset, a visual question answering (VQA) dataset based on SNG that aligns global and local planning. Additionally, we introduce an efficient model SNG-VLA that fuses local planning with global planning. The SNG-VLA achieves state-of-the-art performance through precise navigation information modeling without requiring auxiliary loss functions from perception tasks. Project page: SNG-VLA