Deep Reinforcement Learning for Local Path Following of an Autonomous Formula SAE Vehicle

TL;DR

This paper explores the application of deep reinforcement learning and inverse reinforcement learning to enable autonomous Formula SAE vehicles to follow race tracks locally, using cone detection and steering control in simulation and real-world tests.

Contribution

It introduces the use of SAC and AIRL algorithms for autonomous racing, along with novel reward functions, demonstrating successful local path following in simulation and real-world environments.

Findings

Both algorithms successfully trained models for local path following.

Simulation results translated effectively to real-world vehicle tests.

Proposed reward functions improved learning efficiency.

Abstract

With the continued introduction of driverless events to Formula:Society of Automotive Engineers (F:SAE) competitions around the world, teams are investigating all aspects of the autonomous vehicle stack. This paper presents the use of Deep Reinforcement Learning (DRL) and Inverse Reinforcement Learning (IRL) to map locally-observed cone positions to a desired steering angle for race track following. Two state-of-the-art algorithms not previously tested in this context: soft actor critic (SAC) and adversarial inverse reinforcement learning (AIRL), are used to train models in a representative simulation. Three novel reward functions for use by RL algorithms in an autonomous racing context are also discussed. Tests performed in simulation and the real world suggest that both algorithms can successfully train models for local path following. Suggestions for future work are presented to allow these models to scale to a full F:SAE vehicle.