Using Diffusion Ensembles to Estimate Uncertainty for End-to-End Autonomous Driving

TL;DR

This paper introduces EnDfuser, a diffusion ensemble-based end-to-end autonomous driving system that models trajectory uncertainty, providing interpretability and improved safety in decision-making.

Contribution

EnDfuser is the first to use diffusion ensembles for trajectory planning in autonomous driving, integrating perception and uncertainty modeling into a single transformer-based framework.

Findings

Achieved a driving score of 70.1 on CARLA's Longest6 benchmark.

Produced 128 candidate trajectories per perception frame, capturing multi-modal uncertainties.

Demonstrated that ensemble diffusion improves safety by modeling trajectory distribution uncertainty.

Abstract

End-to-end planning systems for autonomous driving are improving rapidly, especially in closed-loop simulation environments like CARLA. Many such driving systems either do not consider uncertainty as part of the plan itself, or obtain it by using specialized representations that do not generalize. In this paper, we propose EnDfuser, an end-to-end driving system that uses a diffusion model as the trajectory planner. EnDfuser effectively leverages complex perception information like fused camera and LiDAR features, through combining attention pooling and trajectory planning into a single diffusion transformer module. Instead of committing to a single plan, EnDfuser produces a distribution of candidate trajectories (128 for our case) from a single perception frame through ensemble diffusion. By observing the full set of candidate trajectories, EnDfuser provides interpretability for uncertain, multi-modal future trajectory spaces, where there are multiple plausible options. EnDfuser achieves a competitive driving score of 70.1 on the Longest6 benchmark in CARLA with minimal concessions on inference speed. Our findings suggest that ensemble diffusion, used as a drop-in replacement for traditional point-estimate trajectory planning modules, can help improve the safety of driving decisions by modeling the uncertainty of the posterior trajectory distribution.