High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learning

TL;DR

This paper introduces trajectory-aided deep reinforcement learning (TAL) for high-speed autonomous racing, enabling agents to effectively incorporate optimal trajectories and achieve higher lap completion rates at high speeds.

Contribution

The paper proposes a novel trajectory-aided learning method that integrates optimal racing lines into DRL training for high-performance autonomous racing.

Findings

TAL achieves higher lap completion rates at high speeds.

The method enables feasible speed profiling in corners.

TAL outperforms baseline DRL approaches in the F1Tenth simulator.

Abstract

The classical method of autonomous racing uses real-time localisation to follow a precalculated optimal trajectory. In contrast, end-to-end deep reinforcement learning (DRL) can train agents to race using only raw LiDAR scans. While classical methods prioritise optimization for high-performance racing, DRL approaches have focused on low-performance contexts with little consideration of the speed profile. This work addresses the problem of using end-to-end DRL agents for high-speed autonomous racing. We present trajectory-aided learning (TAL) that trains DRL agents for high-performance racing by incorporating the optimal trajectory (racing line) into the learning formulation. Our method is evaluated using the TD3 algorithm on four maps in the open-source F1Tenth simulator. The results demonstrate that our method achieves a significantly higher lap completion rate at high speeds compared to the baseline. This is due to TAL training the agent to select a feasible speed profile of slowing down in the corners and roughly tracking the optimal trajectory.