Robust Autonomy Emerges from Self-Play

TL;DR

This paper demonstrates that self-play in simulation can produce robust, realistic autonomous driving policies that outperform previous methods and achieve high safety and robustness without using human data during training.

Contribution

It introduces a scalable self-play approach for autonomous driving that yields state-of-the-art performance and robustness in simulation and real-world scenarios.

Findings

Achieved 1.6 billion km of simulated driving data.

Outperformed prior state-of-the-art on three benchmarks.

Robustness equivalent to 17.5 years of continuous driving between incidents.

Abstract

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale -- 1.6~billion~km of driving. This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour on a single 8-GPU node. The resulting policy achieves state-of-the-art performance on three independent autonomous driving benchmarks. The policy outperforms the prior state of the art when tested on recorded real-world scenarios, amidst human drivers, without ever seeing human data during training. The policy is realistic when assessed against human references and achieves unprecedented robustness, averaging 17.5 years of continuous driving between incidents in simulation.