MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion

TL;DR

MotionDiffuser introduces a diffusion-based model for multi-agent trajectory prediction that captures diverse outcomes, enables controlled sampling, and achieves state-of-the-art results on the Waymo dataset.

Contribution

The paper presents a novel diffusion-based approach for joint multi-agent motion prediction with a simple training objective and a flexible constrained sampling framework.

Findings

Achieves state-of-the-art performance on Waymo dataset.

Models highly multimodal and joint distributions of trajectories.

Enables rule-based and scenario-specific trajectory sampling.

Abstract

We present MotionDiffuser, a diffusion based representation for the joint distribution of future trajectories over multiple agents. Such representation has several key advantages: first, our model learns a highly multimodal distribution that captures diverse future outcomes. Second, the simple predictor design requires only a single L2 loss training objective, and does not depend on trajectory anchors. Third, our model is capable of learning the joint distribution for the motion of multiple agents in a permutation-invariant manner. Furthermore, we utilize a compressed trajectory representation via PCA, which improves model performance and allows for efficient computation of the exact sample log probability. Subsequently, we propose a general constrained sampling framework that enables controlled trajectory sampling based on differentiable cost functions. This strategy enables a host of applications such as enforcing rules and physical priors, or creating tailored simulation scenarios. MotionDiffuser can be combined with existing backbone architectures to achieve top motion forecasting results. We obtain state-of-the-art results for multi-agent motion prediction on the Waymo Open Motion Dataset.