Acquire Driving Scenarios Efficiently: A Framework for Prospective Assessment of Cost-Optimal Scenario Acquisition

TL;DR

This paper introduces a framework to evaluate and optimize the cost-effectiveness of scenario generation methods for comprehensive testing of automated driving systems, balancing coverage, quality, and resources.

Contribution

It presents a novel methodology to predict scenario coverage and costs, enabling the selection of optimal scenario generation strategies under various constraints.

Findings

Hybrid generation models can achieve coverage efficiently.

The methodology accurately predicts scenario coverage and costs.

Hybrid models outperform pure real-world data collection methods.

Abstract

Scenario-based testing is becoming increasingly important in safety assurance for automated driving. However, comprehensive and sufficiently complete coverage of the scenario space requires significant effort and resources if using only real-world data. To address this issue, driving scenario generation methods are developed and used more frequently, but the benefit of substituting generated data for real-world data has not yet been quantified. Additionally, the coverage of a set of concrete scenarios within a given logical scenario space has not been predicted yet. This paper proposes a methodology to quantify the cost-optimal usage of scenario generation approaches to reach a certainly complete scenario space coverage under given quality constraints and parametrization. Therefore, individual process steps for scenario generation and usage are investigated and evaluated using a meta model for the abstraction of knowledge-based and data-driven methods. Furthermore, a methodology is proposed to fit the meta model including the prediction of reachable complete coverage, quality criteria, and costs. Finally, the paper exemplary examines the suitability of a hybrid generation model under technical, economical, and quality constraints in comparison to different real-world scenario mining methods.