Explainable Reinforcement Learning for Formula One Race Strategy

TL;DR

This paper introduces RSRL, a reinforcement learning model for Formula One race strategy optimization, outperforming traditional methods in simulations and enhancing interpretability and generalizability across tracks.

Contribution

The paper presents a novel RL-based approach for race strategy control that is faster, more adaptable, and interpretable compared to industry-standard methods.

Findings

RSRL achieves an average finishing position of P5.33, outperforming the baseline of P5.63.

The model generalizes well across different tracks and conditions.

Supplementary interpretability tools improve user trust in the model.

Abstract

In Formula One, teams compete to develop their cars and achieve the highest possible finishing position in each race. During a race, however, teams are unable to alter the car, so they must improve their cars' finishing positions via race strategy, i.e. optimising their selection of which tyre compounds to put on the car and when to do so. In this work, we introduce a reinforcement learning model, RSRL (Race Strategy Reinforcement Learning), to control race strategies in simulations, offering a faster alternative to the industry standard of hard-coded and Monte Carlo-based race strategies. Controlling cars with a pace equating to an expected finishing position of P5.5 (where P1 represents first place and P20 is last place), RSRL achieves an average finishing position of P5.33 on our test race, the 2023 Bahrain Grand Prix, outperforming the best baseline of P5.63. We then demonstrate, in a generalisability study, how performance for one track or multiple tracks can be prioritised via training. Further, we supplement model predictions with feature importance, decision tree-based surrogate models, and decision tree counterfactuals towards improving user trust in the model. Finally, we provide illustrations which exemplify our approach in real-world situations, drawing parallels between simulations and reality.