Learning to Drive via Asymmetric Self-Play

TL;DR

This paper introduces asymmetric self-play for autonomous driving, generating synthetic scenarios to improve policy learning, reduce collisions, and enhance transferability without relying solely on real data.

Contribution

It presents a novel asymmetric self-play method that creates challenging synthetic scenarios to train more robust driving policies.

Findings

Reduces collisions in traffic simulation

Improves zero-shot transfer performance

Outperforms state-of-the-art adversarial methods

Abstract

Large-scale data is crucial for learning realistic and capable driving policies. However, it can be impractical to rely on scaling datasets with real data alone. The majority of driving data is uninteresting, and deliberately collecting new long-tail scenarios is expensive and unsafe. We propose asymmetric self-play to scale beyond real data with additional challenging, solvable, and realistic synthetic scenarios. Our approach pairs a teacher that learns to generate scenarios it can solve but the student cannot, with a student that learns to solve them. When applied to traffic simulation, we learn realistic policies with significantly fewer collisions in both nominal and long-tail scenarios. Our policies further zero-shot transfer to generate training data for end-to-end autonomy, significantly outperforming state-of-the-art adversarial approaches, or using real data alone. For more information, visit https://waabi.ai/selfplay .