On the Choice of Data for Efficient Training and Validation of End-to-End Driving Models

TL;DR

This paper examines how data design choices impact the training and validation of end-to-end driving models, providing insights and recommendations for effective data generation and validation strategies in autonomous driving.

Contribution

It systematically analyzes the effects of training data quantity, validation design, and non-determinism on model performance, offering practical guidelines for data management in autonomous driving.

Findings

More training data generally improves driving performance.

Validation design significantly affects generalization to new environments.

Non-determinism influences the consistency of reported improvements.

Abstract

The emergence of data-driven machine learning (ML) has facilitated significant progress in many complicated tasks such as highly-automated driving. While much effort is put into improving the ML models and learning algorithms in such applications, little focus is put into how the training data and/or validation setting should be designed. In this paper we investigate the influence of several data design choices regarding training and validation of deep driving models trainable in an end-to-end fashion. Specifically, (i) we investigate how the amount of training data influences the final driving performance, and which performance limitations are induced through currently used mechanisms to generate training data. (ii) Further, we show by correlation analysis, which validation design enables the driving performance measured during validation to generalize well to unknown test environments. (iii) Finally, we investigate the effect of random seeding and non-determinism, giving insights which reported improvements can be deemed significant. Our evaluations using the popular CARLA simulator provide recommendations regarding data generation and driving route selection for an efficient future development of end-to-end driving models.