Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Experts

TL;DR

This paper introduces a neural network model that predicts multiple future trajectories for autonomous vehicles and other agents, enabling safer planning by selecting trajectories that optimize safety and comfort in real-world driving.

Contribution

It presents a unified neural network architecture for joint prediction and planning using a mixture of experts, improving safety and simplicity without rule-based components.

Findings

The predicted trajectory distribution aligns with different driving profiles.

The method drives safely on urban roads without sacrificing comfort.

Performance improves with more training data.

Abstract

The goal of autonomous vehicles is to navigate public roads safely and comfortably. To enforce safety, traditional planning approaches rely on handcrafted rules to generate trajectories. Machine learning-based systems, on the other hand, scale with data and are able to learn more complex behaviors. However, they often ignore that agents and self-driving vehicle trajectory distributions can be leveraged to improve safety. In this paper, we propose modeling a distribution over multiple future trajectories for both the self-driving vehicle and other road agents, using a unified neural network architecture for prediction and planning. During inference, we select the planning trajectory that minimizes a cost taking into account safety and the predicted probabilities. Our approach does not depend on any rule-based planners for trajectory generation or optimization, improves with more training data and is simple to implement. We extensively evaluate our method through a realistic simulator and show that the predicted trajectory distribution corresponds to different driving profiles. We also successfully deploy it on a self-driving vehicle on urban public roads, confirming that it drives safely without compromising comfort. The code for training and testing our model on a public prediction dataset and the video of the road test are available at https://woven.mobi/safepathnet