SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving

TL;DR

SMARTS is a scalable simulation platform designed to facilitate research in multi-agent reinforcement learning for autonomous driving by providing diverse, realistic interaction scenarios and open-source tools.

Contribution

The paper introduces SMARTS, a novel, scalable multi-agent simulation platform with open-source code and benchmark tasks for advancing autonomous driving research.

Findings

Supports diverse behavior models of road users

Enables realistic multi-agent interaction experiments

Open-sourced for community research

Abstract

Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Despite more than a decade of research and development, the problem of how to competently interact with diverse road users in diverse scenarios remains largely unsolved. Learning methods have much to offer towards solving this problem. But they require a realistic multi-agent simulator that generates diverse and competent driving interactions. To meet this need, we develop a dedicated simulation platform called SMARTS (Scalable Multi-Agent RL Training School). SMARTS supports the training, accumulation, and use of diverse behavior models of road users. These are in turn used to create increasingly more realistic and diverse interactions that enable deeper and broader research on multi-agent interaction. In this paper, we describe the design goals of SMARTS, explain its basic architecture and its key features, and illustrate its use through concrete multi-agent experiments on interactive scenarios. We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving. Our code is available at https://github.com/huawei-noah/SMARTS.