Self driving algorithm for an active four wheel drive racecar

TL;DR

This paper presents a deep reinforcement learning approach using PPO to develop an end-to-end control policy for an active four-wheel-drive racecar, optimizing handling and lap times at the vehicle's handling limits in simulation.

Contribution

It introduces a novel DRL-based control method that directly maps vehicle states to wheel torque and steering commands, bypassing traditional physics-based models and explicit torque vectoring algorithms.

Findings

RL agent learns sophisticated torque distribution strategies.

Achieves competitive lap times in simulation.

Potential to outperform traditional controllers in grip-limited scenarios.

Abstract

Controlling autonomous vehicles at their handling limits is a significant challenge, particularly for electric vehicles with active four wheel drive (A4WD) systems offering independent wheel torque control. While traditional Vehicle Dynamics Control (VDC) methods use complex physics-based models, this study explores Deep Reinforcement Learning (DRL) to develop a unified, high-performance controller. We employ the Proximal Policy Optimization (PPO) algorithm to train an agent for optimal lap times in a simulated racecar (TORCS) at the tire grip limit. Critically, the agent learns an end-to-end policy that directly maps vehicle states, like velocities, accelerations, and yaw rate, to a steering angle command and independent torque commands for each of the four wheels. This formulation bypasses conventional pedal inputs and explicit torque vectoring algorithms, allowing the agent to implicitly learn the A4WD control logic needed for maximizing performance and stability. Simulation results demonstrate the RL agent learns sophisticated strategies, dynamically optimizing wheel torque distribution corner-by-corner to enhance handling and mitigate the vehicle's inherent understeer. The learned behaviors mimic and, in aspects of grip utilization, potentially surpass traditional physics-based A4WD controllers while achieving competitive lap times. This research underscores DRL's potential to create adaptive control systems for complex vehicle dynamics, suggesting RL is a potent alternative for advancing autonomous driving in demanding, grip-limited scenarios for racing and road safety.