Autonomous Drifting Based on Maximal Safety Probability Learning

TL;DR

This paper introduces a physics-informed reinforcement learning framework for autonomous drifting that maximizes safety probability without complex reward shaping, demonstrated in lane keeping and high-speed drifting scenarios.

Contribution

It presents a novel physics-based RL approach that learns maximally safe policies from sparse binary rewards, eliminating the need for trajectory references or reward engineering.

Findings

Successfully learned safe drifting behaviors in racing scenarios

Achieved lane keeping without complex reward shaping

Demonstrated efficiency of physics-informed RL in safety-critical tasks

Abstract

This paper proposes a novel learning-based framework for autonomous driving based on the concept of maximal safety probability. Efficient learning requires rewards that are informative of desirable/undesirable states, but such rewards are challenging to design manually due to the difficulty of differentiating better states among many safe states. On the other hand, learning policies that maximize safety probability does not require laborious reward shaping but is numerically challenging because the algorithms must optimize policies based on binary rewards sparse in time. Here, we show that physics-informed reinforcement learning can efficiently learn this form of maximally safe policy. Unlike existing drift control methods, our approach does not require a specific reference trajectory or complex reward shaping, and can learn safe behaviors only from sparse binary rewards. This is enabled by the use of the physics loss that plays an analogous role to reward shaping. The effectiveness of the proposed approach is demonstrated through lane keeping in a normal cornering scenario and safe drifting in a high-speed racing scenario.