How to Evaluate Proving Grounds for Self-Driving? A Quantitative Approach

TL;DR

This paper introduces a systematic, sample-based method to evaluate the effectiveness of CAV proving grounds in representing real-world traffic scenarios, linking testing results to public street performance.

Contribution

It presents the first comprehensive approach to quantitatively assess proving grounds' scenario coverage and their predictive value for real-world CAV performance.

Findings

Successful evaluation of three world-class proving grounds

Proving ground effectiveness correlates with real-world performance

Benchmark results demonstrate scenario coverage capabilities

Abstract

Proving ground has been a critical component in testing and validation for Connected and Automated Vehicles (CAV). Although quite a few world-class testing facilities have been under construction over the years, the evaluation of proving grounds themselves as testing approaches has rarely been studied. In this paper, we present the first attempt to systematically evaluate CAV proving grounds and contribute to a generative sample-based approach to assessing the representation of traffic scenarios in proving grounds. Leveraging typical use cases extracted from naturalistic driving events, we establish a strong link between proving ground testing results of CAVs and their anticipated public street performance. We present benchmark results of our approach on three world-class CAV testing facilities: Mcity, Almono (Uber ATG), and Kcity. We successfully show the overall evaluation of these proving grounds in terms of their capability to accommodate real-world traffic scenarios. We believe that when the effectiveness of a testing ground itself is validated, the testing results would grant more confidence for CAV public deployment.