Joint Falsification and Fidelity Settings Optimization for Validation of Safety-Critical Systems: A Theoretical Analysis

TL;DR

This paper introduces a theoretical framework for optimizing both falsification and simulator fidelity to improve safety validation of autonomous systems, balancing testing accuracy and computational efficiency.

Contribution

It presents a novel joint optimization method combining counterexample search with fidelity adjustment, supported by rigorous theorems on sensitivity, complexity, and convergence.

Findings

The framework improves testing efficiency by focusing on critical environmental configurations.

Theoretical guarantees include convergence and regret bounds for the optimization process.

The approach enhances confidence in safety validation while reducing computational costs.

Abstract

Safety validation is a crucial component in the development and deployment of autonomous systems, such as self-driving vehicles and robotic systems. Ensuring safe operation necessitates extensive testing and verification of control policies, typically conducted in simulation environments. High-fidelity simulators accurately model real-world dynamics but entail high computational costs, limiting their scalability for exhaustive testing. Conversely, low-fidelity simulators offer efficiency but may not capture the intricacies of high-fidelity simulators, potentially yielding false conclusions. We propose a joint falsification and fidelity optimization framework for safety validation of autonomous systems. Our mathematical formulation combines counterexample searches with simulator fidelity improvement, facilitating more efficient exploration of the critical environmental configurations challenging the control system. Our contributions encompass a set of theorems addressing counterexample sensitivity analysis, sample complexity, convergence, the interplay between the outer and inner optimization loops, and regret bound analysis. The proposed joint optimization approach enables a more targeted and efficient testing process, optimizes the use of available computational resources, and enhances confidence in autonomous system safety validation.