Transfer Importance Sampling -- How Testing Automated Vehicles in Multiple Test Setups Helps With the Bias-Variance Tradeoff

TL;DR

This paper introduces transfer importance sampling (TIS), a novel method combining transfer learning and importance sampling to efficiently and accurately estimate risks of automated vehicles across different testing environments, balancing cost and reliability.

Contribution

The paper proposes transfer importance sampling (TIS), a new approach that links virtual and real test setups to improve risk estimation for automated vehicles, addressing cost and bias-variance tradeoffs.

Findings

TIS effectively combines virtual and real tests for better risk estimation.

Linking test setups improves efficiency without losing accuracy.

TIS reduces costs in AV testing by leveraging scalable test environments.

Abstract

The promise of increased road safety is a key motivator for the development of automated vehicles (AV). Yet, demonstrating that an AV is as safe as, or even safer than, a human-driven vehicle has proven to be challenging. Should an AV be examined purely virtually, allowing large numbers of fully controllable tests? Or should it be tested under real environmental conditions on a proving ground? Since different test setups have different strengths and weaknesses, it is still an open question how virtual and real tests should be combined. On the way to answer this question, this paper proposes transfer importance sampling (TIS), a risk estimation method linking different test setups. Fusing the concepts of transfer learning and importance sampling, TIS uses a scalable, cost-effective test setup to comprehensively explore an AV's behavior. The insights gained then allow parameterizing tests in a more trustworthy test setup accurately reflecting risks. We show that when using a trustworthy test setup alone is prohibitively expensive, linking it to a scalable test setup can increase efficiency $\unicode{x2013}$ without sacrificing the result's validity. Thus, the test setups' individual deficiencies are compensated for by their systematic linkage.