Practical Equivalence Testing and Its Application in Synthetic Pre-Crash Scenario Validation

TL;DR

This paper introduces a Bayesian equivalence testing method to validate the similarity of synthetic pre-crash scenarios to real-world data, enhancing the credibility of virtual safety assessments for automated vehicles.

Contribution

It proposes a novel Bayesian ROPE-based equivalence testing approach tailored for validating synthetic pre-crash scenarios, addressing limitations of traditional significance tests.

Findings

The Bayesian ROPE method effectively assesses scenario equivalence.

Application to rear-end crash datasets demonstrates practical utility.

Highlights importance of equivalence testing for synthetic data validation.

Abstract

The use of representative pre-crash scenarios is critical for assessing the safety impact of driving automation systems through simulation. However, a gap remains in the robust evaluation of the similarity between synthetic and real-world pre-crash scenarios and their crash characteristics. Without proper validation, it cannot be ensured that the synthetic test scenarios adequately represent real-world driving behaviors and crash characteristics. One reason for this validation gap is the lack of focus on methods to confirm that the synthetic test scenarios are practically equivalent to real-world ones, given the assessment scope. Traditional statistical methods, like significance testing, focus on detecting differences rather than establishing equivalence; since failure to detect a difference does not imply equivalence, they are of limited applicability for validating synthetic pre-crash scenarios and crash characteristics. This study addresses this gap by proposing an equivalence testing method based on the Bayesian Region of Practical Equivalence (ROPE) framework. This method is designed to assess the practical equivalence of scenario characteristics that are most relevant for the intended assessment, making it particularly appropriate for the domain of virtual safety assessments. We first review existing equivalence testing methods. Then we propose and demonstrate the Bayesian ROPE-based method by testing the equivalence of two rear-end pre-crash datasets. Our approach focuses on the most relevant scenario characteristics. Our analysis provides insights into the practicalities and effectiveness of equivalence testing in synthetic test scenario validation and demonstrates the importance of testing for improving the credibility of synthetic data for automated vehicle safety assessment, as well as the credibility of subsequent safety impact assessments.