Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Driving

TL;DR

This paper introduces LatentDriver, a probabilistic framework for autonomous driving that models environment states and actions as a mixture distribution, improving decision-making and generalization over existing methods.

Contribution

The paper proposes LatentDriver, a novel approach that captures decision stochasticity and mitigates self-delusion by using mixture modeling and intermediate action sampling.

Findings

LatentDriver outperforms state-of-the-art reinforcement and imitation learning methods.

Achieves expert-level performance on the Waymax benchmark.

Provides a robust probabilistic decision-making framework for autonomous driving.

Abstract

The autoregressive world model exhibits robust generalization capabilities in vectorized scene understanding but encounters difficulties in deriving actions due to insufficient uncertainty modeling and self-delusion. In this paper, we explore the feasibility of deriving decisions from an autoregressive world model by addressing these challenges through the formulation of multiple probabilistic hypotheses. We propose LatentDriver, a framework models the environment's next states and the ego vehicle's possible actions as a mixture distribution, from which a deterministic control signal is then derived. By incorporating mixture modeling, the stochastic nature of decisionmaking is captured. Additionally, the self-delusion problem is mitigated by providing intermediate actions sampled from a distribution to the world model. Experimental results on the recently released close-loop benchmark Waymax demonstrate that LatentDriver surpasses state-of-the-art reinforcement learning and imitation learning methods, achieving expert-level performance. The code and models will be made available at https://github.com/Sephirex-X/LatentDriver.