SUPER-AD: Semantic Uncertainty-aware Planning for End-to-End Robust Autonomous Driving

TL;DR

This paper introduces a novel camera-only end-to-end autonomous driving framework that explicitly models aleatoric uncertainty in BEV space, improving safety and robustness in uncertain scenarios.

Contribution

It presents a new uncertainty-aware planning method with a dense drivability map and lane-following regularization, enhancing robustness and interpretability in autonomous driving.

Findings

Achieves state-of-the-art results on NAVSIM benchmark.

Significantly improves safety and reliability under uncertainty.

Effectively incorporates semantic and geometric information at pixel-level.

Abstract

End-to-End (E2E) planning has become a powerful paradigm for autonomous driving, yet current systems remain fundamentally uncertainty-blind. They assume perception outputs are fully reliable, even in ambiguous or poorly observed scenes, leaving the planner without an explicit measure of uncertainty. To address this limitation, we propose a camera-only E2E framework that estimates aleatoric uncertainty directly in BEV space and incorporates it into planning. Our method produces a dense, uncertainty-aware drivability map that captures both semantic structure and geometric layout at pixel-level resolution. To further promote safe and rule-compliant behavior, we introduce a lane-following regularization that encodes lane structure and traffic norms. This prior stabilizes trajectory planning under normal conditions while preserving the flexibility needed for maneuvers such as overtaking or lane changes. Together, these components enable robust and interpretable trajectory planning, even under challenging uncertainty conditions. Evaluated on the NAVSIM benchmark, our method achieves state-of-the-art performance, delivering substantial gains on both the challenging NAVHARD and NAVSAFE subsets. These results demonstrate that our principled aleatoric uncertainty modeling combined with driving priors significantly advances the safety and reliability of camera-only E2E autonomous driving.