TrajDiff: End-to-end Autonomous Driving without Perception Annotation

TL;DR

TrajDiff introduces a perception annotation-free end-to-end autonomous driving framework that generates diverse driving trajectories directly from raw sensor data, achieving state-of-the-art performance without manual perception labels.

Contribution

The paper presents TrajDiff, a novel perception annotation-free generative model for autonomous driving that leverages BEV heatmaps and diffusion transformers to produce plausible trajectories.

Findings

Achieves 87.5 PDMS on NAVSIM benchmark.

Further improves to 88.5 PDMS with data scaling.

Eliminates need for perception annotations and handcrafted motion priors.

Abstract

End-to-end autonomous driving systems directly generate driving policies from raw sensor inputs. While these systems can extract effective environmental features for planning, relying on auxiliary perception tasks, developing perception annotation-free planning paradigms has become increasingly critical due to the high cost of manual perception annotation. In this work, we propose TrajDiff, a Trajectory-oriented BEV Conditioned Diffusion framework that establishes a fully perception annotation-free generative method for end-to-end autonomous driving. TrajDiff requires only raw sensor inputs and future trajectory, constructing Gaussian BEV heatmap targets that inherently capture driving modalities. We design a simple yet effective trajectory-oriented BEV encoder to extract the TrajBEV feature without perceptual supervision. Furthermore, we introduce Trajectory-oriented BEV Diffusion Transformer (TB-DiT), which leverages ego-state information and the predicted TrajBEV features to directly generate diverse yet plausible trajectories, eliminating the need for handcrafted motion priors. Beyond architectural innovations, TrajDiff enables exploration of data scaling benefits in the annotation-free setting. Evaluated on the NAVSIM benchmark, TrajDiff achieves 87.5 PDMS, establishing state-of-the-art performance among all annotation-free methods. With data scaling, it further improves to 88.5 PDMS, which is comparable to advanced perception-based approaches. Our code and model will be made publicly available.