First-principles Based 3D Virtual Simulation Testing for Discovering SOTIF Corner Cases of Autonomous Driving

TL;DR

This paper introduces a first-principles sensor modeling approach integrated into a simulation platform to discover weather-related corner cases in autonomous driving, significantly improving the detection efficiency over existing methods.

Contribution

It presents a novel first-principles based sensor and environment interaction scheme and a meta-heuristic algorithm to efficiently identify corner cases in autonomous driving simulations.

Findings

Discovered new weather-related corner cases with root causes.

Achieved four times more corner case discoveries than state-of-the-art methods.

Enhanced simulation accuracy by modeling sensor physics and environment interactions.

Abstract

3D virtual simulation, which generates diversified test scenarios and tests full-stack of Autonomous Driving Systems (ADSes) modules dynamically as a whole, is a promising approach for Safety of The Intended Functionality (SOTIF) ADS testing. However, as different configurations of a test scenario will affect the sensor perceptions and environment interaction, e.g. light pulses emitted by the LiDAR sensor will undergo backscattering and attenuation, which is usually overlooked by existing works, leading to false positives or wrong results. Moreover, the input space of an ADS is extremely large, with infinite number of possible initial scenarios and mutations, along both temporal and spatial domains. This paper proposes a first-principles based sensor modeling and environment interaction scheme, and integrates it into CARLA simulator. With this scheme, a long-overlooked category of adverse weather related corner cases are discovered, along with their root causes. Moreover, a meta-heuristic algorithm is designed based on several empirical insights, which guide both seed scenarios and mutations, significantly reducing the search dimensions of scenarios and enhancing the efficiency of corner case identification. Experimental results show that under identical simulation setups, our algorithm discovers about four times as many corner cases as compared to state-of-the-art work.