Learn-to-Race: A Multimodal Control Environment for Autonomous Racing

TL;DR

This paper introduces Learn-to-Race, a comprehensive multimodal simulation environment for autonomous racing, including a new simulator, dataset, and benchmarks to advance high-speed autonomous driving research.

Contribution

The paper presents the Arrival simulator, a new multimodal racing environment, and establishes the L2R task with benchmarks, datasets, and baseline experiments for autonomous racing research.

Findings

Arrival simulator models vehicle dynamics accurately

L2R benchmarks challenge multimodal autonomous racing

Baseline experiments provide reference performance levels

Abstract

Existing research on autonomous driving primarily focuses on urban driving, which is insufficient for characterising the complex driving behaviour underlying high-speed racing. At the same time, existing racing simulation frameworks struggle in capturing realism, with respect to visual rendering, vehicular dynamics, and task objectives, inhibiting the transfer of learning agents to real-world contexts. We introduce a new environment, where agents Learn-to-Race (L2R) in simulated competition-style racing, using multimodal information--from virtual cameras to a comprehensive array of inertial measurement sensors. Our environment, which includes a simulator and an interfacing training framework, accurately models vehicle dynamics and racing conditions. In this paper, we release the Arrival simulator for autonomous racing. Next, we propose the L2R task with challenging metrics, inspired by learning-to-drive challenges, Formula-style racing, and multimodal trajectory prediction for autonomous driving. Additionally, we provide the L2R framework suite, facilitating simulated racing on high-precision models of real-world tracks. Finally, we provide an official L2R task dataset of expert demonstrations, as well as a series of baseline experiments and reference implementations. We make all code available: https://github.com/learn-to-race/l2r.