Learning Autonomous Race Driving with Action Mapping Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach with action mapping for autonomous race driving, effectively handling tire-road friction constraints and generalizing across different conditions, resulting in improved lap times and success rates.

Contribution

The paper presents a novel RL method with action mapping to manage state-dependent constraints and enhance policy generalization in autonomous racing.

Findings

Achieves superior lap times compared to conventional RL methods.

Demonstrates effective generalization across different friction conditions.

Improves success rates in autonomous race driving simulations.

Abstract

Autonomous race driving poses a complex control challenge as vehicles must be operated at the edge of their handling limits to reduce lap times while respecting physical and safety constraints. This paper presents a novel reinforcement learning (RL)-based approach, incorporating the action mapping (AM) mechanism to manage state-dependent input constraints arising from limited tire-road friction. A numerical approximation method is proposed to implement AM, addressing the complex dynamics associated with the friction constraints. The AM mechanism also allows the learned driving policy to be generalized to different friction conditions. Experimental results in our developed race simulator demonstrate that the proposed AM-RL approach achieves superior lap times and better success rates compared to the conventional RL-based approaches. The generalization capability of driving policy with AM is also validated in the experiments.