Generalized Trajectory Scoring for End-to-end Multimodal Planning

TL;DR

GTRS introduces a unified trajectory scoring framework combining diffusion-based proposals, vocabulary generalization, and sensor augmentation to improve end-to-end multimodal planning in autonomous driving, especially under challenging conditions.

Contribution

It presents a novel integrated approach that enhances trajectory scoring robustness and generalization in autonomous driving, outperforming existing methods.

Findings

Won the Navsim v2 Challenge with superior performance.

Demonstrated robustness with sub-optimal sensor inputs.

Approached privileged methods relying on ground-truth perception.

Abstract

End-to-end multi-modal planning is a promising paradigm in autonomous driving, enabling decision-making with diverse trajectory candidates. A key component is a robust trajectory scorer capable of selecting the optimal trajectory from these candidates. While recent trajectory scorers focus on scoring either large sets of static trajectories or small sets of dynamically generated ones, both approaches face significant limitations in generalization. Static vocabularies provide effective coarse discretization but struggle to make fine-grained adaptation, while dynamic proposals offer detailed precision but fail to capture broader trajectory distributions. To overcome these challenges, we propose GTRS (Generalized Trajectory Scoring), a unified framework for end-to-end multi-modal planning that combines coarse and fine-grained trajectory evaluation. GTRS consists of three complementary innovations: (1) a diffusion-based trajectory generator that produces diverse fine-grained proposals; (2) a vocabulary generalization technique that trains a scorer on super-dense trajectory sets with dropout regularization, enabling its robust inference on smaller subsets; and (3) a sensor augmentation strategy that enhances out-of-domain generalization while incorporating refinement training for critical trajectory discrimination. As the winning solution of the Navsim v2 Challenge, GTRS demonstrates superior performance even with sub-optimal sensor inputs, approaching privileged methods that rely on ground-truth perception. Code will be available at https://github.com/NVlabs/GTRS.