RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms

TL;DR

RLPP is a hybrid residual reinforcement learning framework that enhances a traditional Pure Pursuit controller with RL, significantly improving real-world autonomous racing performance and reducing the sim-to-real gap.

Contribution

The paper introduces RLPP, a novel residual RL approach that combines classical control with RL to improve zero-shot real-world autonomous racing performance.

Findings

RLPP improves lap times by up to 6.37%

Reduces the sim-to-real gap by more than 8-fold

Closes over 52% of the performance gap to state-of-the-art methods

Abstract

Autonomous racing presents a complex environment requiring robust controllers capable of making rapid decisions under dynamic conditions. While traditional controllers based on tire models are reliable, they often demand extensive tuning or system identification. Reinforcement Learning (RL) methods offer significant potential due to their ability to learn directly from interaction, yet they typically suffer from the sim-to-real gap, where policies trained in simulation fail to perform effectively in the real world. In this paper, we propose RLPP, a residual RL framework that enhances a Pure Pursuit (PP) controller with an RL-based residual. This hybrid approach leverages the reliability and interpretability of PP while using RL to fine-tune the controller's performance in real-world scenarios. Extensive testing on the F1TENTH platform demonstrates that RLPP improves lap times of the baseline controllers by up to 6.37 %, closing the gap to the State-of-the-Art methods by more than 52 % and providing reliable performance in zero-shot real-world deployment, overcoming key challenges associated with the sim-to-real transfer and reducing the performance gap from simulation to reality by more than 8-fold when compared to the baseline RL controller. The RLPP framework is made available as an open-source tool, encouraging further exploration and advancement in autonomous racing research. The code is available at: www.github.com/forzaeth/rlpp.