Model Predictive Simulation Using Structured Graphical Models and Transformers

TL;DR

This paper introduces Model Predictive Simulation (MPS), combining transformers and probabilistic graphical models to generate and refine multi-agent trajectories, improving safety metrics without additional training.

Contribution

The paper presents MPS, a novel framework that enhances transformer-based trajectory predictions with PGMs for better safety and adaptability in multi-agent simulation.

Findings

MPS improves collision avoidance over baseline models.

The approach is compatible with any forecasting model.

No extra training required for the PGM component.

Abstract

We propose an approach to simulating trajectories of multiple interacting agents (road users) based on transformers and probabilistic graphical models (PGMs), and apply it to the Waymo SimAgents challenge. The transformer baseline is based on the MTR model, which predicts multiple future trajectories conditioned on the past trajectories and static road layout features. We then improve upon these generated trajectories using a PGM, which contains factors which encode prior knowledge, such as a preference for smooth trajectories, and avoidance of collisions with static obstacles and other moving agents. We perform (approximate) MAP inference in this PGM using the Gauss-Newton method. Finally we sample $K=32$ trajectories for each of the $N \sim 100$ agents for the next $T=8 \Delta$ time steps, where $\Delta=10$ is the sampling rate per second. Following the Model Predictive Control (MPC) paradigm, we only return the first element of our forecasted trajectories at each step, and then we replan, so that the simulation can constantly adapt to its changing environment. We therefore call our approach "Model Predictive Simulation" or MPS. We show that MPS improves upon the MTR baseline, especially in safety critical metrics such as collision rate. Furthermore, our approach is compatible with any underlying forecasting model, and does not require extra training, so we believe it is a valuable contribution to the community.