Planning Reliability Assurance Tests for Autonomous Vehicles

TL;DR

This paper develops statistical methods for planning autonomous vehicle reliability tests using recurrent event data, addressing a gap in AV testing strategies and providing practical recommendations.

Contribution

It introduces novel statistical test planning strategies based on Poisson processes and Pareto front optimization for AV reliability assurance.

Findings

Effective test planning strategies for AV reliability using homogeneous and non-homogeneous Poisson processes.

Application of methods to real AV disengagement data from California DMV.

Guidelines for balancing multiple objectives in AV test planning.

Abstract

Artificial intelligence (AI) technology has become increasingly prevalent and transforms our everyday life. One important application of AI technology is the development of autonomous vehicles (AV). However, the reliability of an AV needs to be carefully demonstrated via an assurance test so that the product can be used with confidence in the field. To plan for an assurance test, one needs to determine how many AVs need to be tested for how many miles and the standard for passing the test. Existing research has made great efforts in developing reliability demonstration tests in the other fields of applications for product development and assessment. However, statistical methods have not been utilized in AV test planning. This paper aims to fill in this gap by developing statistical methods for planning AV reliability assurance tests based on recurrent events data. We explore the relationship between multiple criteria of interest in the context of planning AV reliability assurance tests. Specifically, we develop two test planning strategies based on homogeneous and non-homogeneous Poisson processes while balancing multiple objectives with the Pareto front approach. We also offer recommendations for practical use. The disengagement events data from the California Department of Motor Vehicles AV testing program is used to illustrate the proposed assurance test planning methods.