Addressing the Waypoint-Action Gap in End-to-End Autonomous Driving via Vehicle Motion Models

TL;DR

This paper introduces a differentiable vehicle-model framework that bridges the gap between waypoint-based and action-based end-to-end autonomous driving models, enabling better training and evaluation.

Contribution

It presents a novel vehicle-model framework that allows action-based models to be trained and evaluated within waypoint-based benchmarks without protocol modifications.

Findings

Achieves state-of-the-art performance on NAVSIM navhard benchmark.

Consistent improvements over baseline models across multiple benchmarks.

Enables training of action-based policies within waypoint-based evaluation protocols.

Abstract

End-to-End Autonomous Driving (E2E-AD) systems are typically grouped by the nature of their outputs: (i) waypoint-based models that predict a future trajectory, and (ii) action-based models that directly output throttle, steer and brake. Most recent benchmark protocols and training pipelines are waypoint-based, which makes action-based policies harder to train and compare, slowing their progress. To bridge this waypoint-action gap, we propose a novel, differentiable vehicle-model framework that rolls out predicted action sequences to their corresponding ego-frame waypoint trajectories while supervising in waypoint space. Our approach enables action-based architectures to be trained and evaluated, for the first time, within waypoint-based benchmarks without modifying the underlying evaluation protocol. We extensively evaluate our framework across multiple challenging benchmarks and observe consistent improvements over the baselines. In particular, on NAVSIM \texttt{navhard} our approach achieves state-of-the-art performance. Our code will be made publicly available upon acceptance.