Physics-informed Imitative Reinforcement Learning for Real-world Driving

TL;DR

This paper introduces a physics-informed imitative reinforcement learning method that combines expert demonstrations and exploratory data to improve autonomous driving, significantly reducing collisions and off-road incidents.

Contribution

It presents a novel data-driven IRL approach that incorporates physical principles of vehicle dynamics, enhancing transferability and performance in real-world driving scenarios.

Findings

37.8% reduction in collision rate

22.2% reduction in off-road rate

Outperforms popular IL, RL, and IRL algorithms on Waymax benchmark

Abstract

Recent advances in imitative reinforcement learning (IRL) have considerably enhanced the ability of autonomous agents to assimilate expert demonstrations, leading to rapid skill acquisition in a range of demanding tasks. However, such learning-based agents face significant challenges when transferring knowledge to highly dynamic closed-loop environments. Their performance is significantly impacted by the conflicting optimization objectives of imitation learning (IL) and reinforcement learning (RL), sample inefficiency, and the complexity of uncovering the hidden world model and physics. To address this challenge, we propose a physics-informed IRL that is entirely data-driven. It leverages both expert demonstration data and exploratory data with a joint optimization objective, allowing the underlying physical principles of vehicle dynamics to emerge naturally from the training process. The performance is evaluated through empirical experiments and results exceed popular IL, RL and IRL algorithms in closed-loop settings on Waymax benchmark. Our approach exhibits 37.8% reduction in collision rate and 22.2% reduction in off-road rate compared to the baseline method.