Learn 2 Rage: Experiencing The Emotional Roller Coaster That Is Reinforcement Learning

TL;DR

This paper describes a reinforcement learning approach for autonomous racing that emphasizes visual perception, rule-based control, and modifications to reward functions, leading to a winning system that is efficient, explainable, and highly effective.

Contribution

The authors developed a novel RL-based autonomous racing system using visual features and rule-based controllers, outperforming other agents in a competitive challenge.

Findings

Rule-based controllers with learned visual perception outperform pure pixel-to-action models.

Modified reward policies promote smoother control behaviors.

The system requires less training and offers better generalization.

Abstract

This work presents the experiments and solution outline for our teams winning submission in the Learn To Race Autonomous Racing Virtual Challenge 2022 hosted by AIcrowd. The objective of the Learn-to-Race competition is to push the boundary of autonomous technology, with a focus on achieving the safety benefits of autonomous driving. In the description the competition is framed as a reinforcement learning (RL) challenge. We focused our initial efforts on implementation of Soft Actor Critic (SAC) variants. Our goal was to learn non-trivial control of the race car exclusively from visual and geometric features, directly mapping pixels to control actions. We made suitable modifications to the default reward policy aiming to promote smooth steering and acceleration control. The framework for the competition provided real time simulation, meaning a single episode (learning experience) is measured in minutes. Instead of pursuing parallelisation of episodes we opted to explore a more traditional approach in which the visual perception was processed (via learned operators) and fed into rule-based controllers. Such a system, while not as academically "attractive" as a pixels-to-actions approach, results in a system that requires less training, is more explainable, generalises better and is easily tuned and ultimately out-performed all other agents in the competition by a large margin.