Benchmarking of a software stack for autonomous racing against a professional human race driver

TL;DR

This paper compares a professional race driver's skills with an autonomous racing software stack to identify performance gaps and guide improvements for autonomous vehicle safety and handling in extreme situations.

Contribution

It introduces a benchmarking approach using motorsport data analysis techniques to evaluate and enhance autonomous racing software against human expert performance.

Findings

The software shows promising performance but lags behind the professional driver in handling extreme maneuvers.

Insights from the comparison guide targeted improvements in the autonomous system.

The study informs development strategies for safer, more capable autonomous vehicles.

Abstract

The way to full autonomy of public road vehicles requires the step-by-step replacement of the human driver, with the ultimate goal of replacing the driver completely. Eventually, the driving software has to be able to handle all situations that occur on its own, even emergency situations. These particular situations require extreme combined braking and steering actions at the limits of handling to avoid an accident or to diminish its consequences. An average human driver is not trained to handle such extreme and rarely occurring situations and therefore often fails to do so. However, professional race drivers are trained to drive a vehicle utilizing the maximum amount of possible tire forces. These abilities are of high interest for the development of autonomous driving software. Here, we compare a professional race driver and our software stack developed for autonomous racing with data analysis techniques established in motorsports. The goal of this research is to derive indications for further improvement of the performance of our software and to identify areas where it still fails to meet the performance level of the human race driver. Our results are used to extend our software's capabilities and also to incorporate our findings into the research and development of public road autonomous vehicles.