Multiverse Transformer: 1st Place Solution for Waymo Open Sim Agents Challenge 2023

TL;DR

This paper introduces the Multiverse Transformer (MVTA), a novel transformer-based approach for agent simulation in autonomous driving, achieving first place in the Waymo Open Sim Agents Challenge 2023 with high realism metrics.

Contribution

The paper presents a new transformer architecture tailored for closed-loop agent simulation, including novel training, sampling, and history aggregation methods to improve realism and reduce errors.

Findings

MVTA achieves a realism metric of 0.5091.

Enhanced MVTE reaches 0.5168, outperforming competitors.

Introduces receding horizon prediction and variable-length history aggregation.

Abstract

This technical report presents our 1st place solution for the Waymo Open Sim Agents Challenge (WOSAC) 2023. Our proposed MultiVerse Transformer for Agent simulation (MVTA) effectively leverages transformer-based motion prediction approaches, and is tailored for closed-loop simulation of agents. In order to produce simulations with a high degree of realism, we design novel training and sampling methods, and implement a receding horizon prediction mechanism. In addition, we introduce a variable-length history aggregation method to mitigate the compounding error that can arise during closed-loop autoregressive execution. On the WOSAC, our MVTA and its enhanced version MVTE reach a realism meta-metric of 0.5091 and 0.5168, respectively, outperforming all the other methods on the leaderboard.