Towards Optimal Head-to-head Autonomous Racing with Curriculum Reinforcement Learning

TL;DR

This paper introduces a curriculum reinforcement learning framework with safety mechanisms for autonomous head-to-head racing, enabling the vehicle to learn near-optimal strategies in complex dynamic environments.

Contribution

It presents a curriculum learning approach transitioning from simple to complex vehicle models and a control barrier function-based safe RL algorithm for improved safety and performance.

Findings

Achieved near-optimal racing policies in complex environments

Demonstrated safety and efficiency improvements over baseline methods

Validated the approach in a realistic head-to-head racing simulation

Abstract

Head-to-head autonomous racing is a challenging problem, as the vehicle needs to operate at the friction or handling limits in order to achieve minimum lap times while also actively looking for strategies to overtake/stay ahead of the opponent. In this work we propose a head-to-head racing environment for reinforcement learning which accurately models vehicle dynamics. Some previous works have tried learning a policy directly in the complex vehicle dynamics environment but have failed to learn an optimal policy. In this work, we propose a curriculum learning-based framework by transitioning from a simpler vehicle model to a more complex real environment to teach the reinforcement learning agent a policy closer to the optimal policy. We also propose a control barrier function-based safe reinforcement learning algorithm to enforce the safety of the agent in a more effective way while not compromising on optimality.