Formula RL: Deep Reinforcement Learning for Autonomous Racing using Telemetry Data

TL;DR

This paper investigates deep reinforcement learning models for autonomous racing, demonstrating that RL-trained models outperform handcrafted bots and can generalize to new tracks using telemetry data and continuous actions.

Contribution

It introduces a framework applying various deep deterministic policy gradient methods to autonomous racing with telemetry data, analyzing their learning and generalization capabilities.

Findings

RL models outperform handcrafted bots in lap-time

RL models generalize to unknown tracks

Deep deterministic policy gradient methods are effective for racing

Abstract

This paper explores the use of reinforcement learning (RL) models for autonomous racing. In contrast to passenger cars, where safety is the top priority, a racing car aims to minimize the lap-time. We frame the problem as a reinforcement learning task with a multidimensional input consisting of the vehicle telemetry, and a continuous action space. To find out which RL methods better solve the problem and whether the obtained models generalize to driving on unknown tracks, we put 10 variants of deep deterministic policy gradient (DDPG) to race in two experiments: i)~studying how RL methods learn to drive a racing car and ii)~studying how the learning scenario influences the capability of the models to generalize. Our studies show that models trained with RL are not only able to drive faster than the baseline open source handcrafted bots but also generalize to unknown tracks.