Analyzing the Impact of Simulation Fidelity on the Evaluation of Autonomous Driving Motion Control

TL;DR

This paper investigates how the level of detail in vehicle simulation models affects the evaluation of autonomous driving controllers, using extensive simulation and real-world data from a high-speed race to determine appropriate model fidelity.

Contribution

It introduces a comprehensive vehicle model compatible with Autoware, derives simplified models, and quantifies their accuracy and impact on control algorithm evaluation across different scenarios.

Findings

Simplified models can approximate real vehicle behavior with acceptable accuracy for certain applications.

Model fidelity influences the evaluation of control algorithms, especially near acceleration limits.

The degree of model simplification should be chosen based on the specific evaluation context.

Abstract

Simulation is crucial in the development of autonomous driving software. In particular, assessing control algorithms requires an accurate vehicle dynamics simulation. However, recent publications use models with varying levels of detail. This disparity makes it difficult to compare individual control algorithms. Therefore, this paper aims to investigate the influence of the fidelity of vehicle dynamics modeling on the closed-loop behavior of trajectory-following controllers. For this purpose, we introduce a comprehensive Autoware-compatible vehicle model. By simplifying this, we derive models with varying fidelity. Evaluating over 550 simulation runs allows us to quantify each model's approximation quality compared to real-world data. Furthermore, we investigate whether the influence of model simplifications changes with varying margins to the acceleration limit of the vehicle. From this, we deduce to which degree a vehicle model can be simplified to evaluate control algorithms depending on the specific application. The real-world data used to validate the simulation environment originate from the Indy Autonomous Challenge race at the Autodromo Nazionale di Monza in June 2023. They show the fastest fully autonomous lap of TUM Autonomous Motorsport, with vehicle speeds reaching 267 kph and lateral accelerations of up to 15 mps2.