From Human Intention to Action Prediction: Intention-Driven End-to-End Autonomous Driving

TL;DR

This paper introduces a new benchmark and evaluation protocol for intention-driven autonomous driving, emphasizing semantic understanding of human goals and proposing novel models that better align with human intentions.

Contribution

It defines the task of intention-driven driving, creates a large-scale dataset with natural language intentions, and introduces the Imagined Future Alignment evaluation protocol.

Findings

Existing models are stable but lack intention fulfillment.

Proposed models show improved alignment with human goals.

The benchmark facilitates semantic-aware evaluation of autonomous driving.

Abstract

While end-to-end autonomous driving has achieved remarkable progress in geometric control, current systems remain constrained by a command-following paradigm that relies on simple navigational instructions. Transitioning to genuinely intelligent agents requires the capability to interpret and fulfill high-level, abstract human intentions. However, this advancement is hindered by the lack of dedicated benchmarks and semantic-aware evaluation metrics. In this paper, we formally define the task of Intention-Driven End-to-End Autonomous Driving and present Intention-Drive, a comprehensive benchmark designed to bridge this gap. We construct a large-scale dataset featuring complex natural language intentions paired with high-fidelity sensor data. To overcome the limitations of conventional trajectory-based metrics, we introduce the Imagined Future Alignment (IFA), a novel evaluation protocol leveraging generative world models to assess the semantic fulfillment of human goals beyond mere geometric accuracy. Furthermore, we explore the solution space by proposing two distinct paradigms: an end-to-end vision-language planner and a hierarchical agent-based framework. The experiments reveal a critical dichotomy where existing models exhibit satisfactory driving stability but struggle significantly with intention fulfillment. Notably, the proposed frameworks demonstrate superior alignment with human intentions.