MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning

TL;DR

MetaDrive is a versatile driving simulation platform that enables research on generalizable reinforcement learning by generating diverse scenarios and benchmarking various RL algorithms in single and multi-agent settings.

Contribution

The paper introduces MetaDrive, a highly compositional simulation environment supporting diverse scenario generation and comprehensive RL benchmarking for autonomous driving research.

Findings

Increasing training diversity improves RL generalization.

MetaDrive effectively benchmarks safe and multi-agent RL algorithms.

Diverse scenarios enhance robustness of RL agents.

Abstract

Driving safely requires multiple capabilities from human and intelligent agents, such as the generalizability to unseen environments, the safety awareness of the surrounding traffic, and the decision-making in complex multi-agent settings. Despite the great success of Reinforcement Learning (RL), most of the RL research works investigate each capability separately due to the lack of integrated environments. In this work, we develop a new driving simulation platform called MetaDrive to support the research of generalizable reinforcement learning algorithms for machine autonomy. MetaDrive is highly compositional, which can generate an infinite number of diverse driving scenarios from both the procedural generation and the real data importing. Based on MetaDrive, we construct a variety of RL tasks and baselines in both single-agent and multi-agent settings, including benchmarking generalizability across unseen scenes, safe exploration, and learning multi-agent traffic. The generalization experiments conducted on both procedurally generated scenarios and real-world scenarios show that increasing the diversity and the size of the training set leads to the improvement of the RL agent's generalizability. We further evaluate various safe reinforcement learning and multi-agent reinforcement learning algorithms in MetaDrive environments and provide the benchmarks. Source code, documentation, and demo video are available at \url{ https://metadriverse.github.io/metadrive}.