Enhancing Uncertainty Quantification for Runtime Safety Assurance Using Causal Risk Analysis and Operational Design Domain

TL;DR

This paper enhances uncertainty quantification for autonomous system safety by integrating environmental conditions and causal risk analysis into a Bayesian framework, enabling dynamic, context-aware safety assessments.

Contribution

It introduces a novel method combining hazard analysis, fault trees, and Bayesian networks to improve real-time safety uncertainty estimation in autonomous systems.

Findings

Improved safety uncertainty estimation with environmental context.

Demonstrated effectiveness on an Automated Valet Parking system.

Provides dynamic, real-time safety risk assessments.

Abstract

Ensuring the runtime safety of autonomous systems remains challenging due to deep learning components' inherent uncertainty and their sensitivity to environmental changes. In this paper, we propose an enhancement of traditional uncertainty quantification by explicitly incorporating environmental conditions using risk-based causal analysis. We leverage Hazard Analysis and Risk Assessment (HARA) and fault tree modeling to identify critical operational conditions affecting system functionality. These conditions, together with uncertainties from the data and model, are integrated into a unified Bayesian Network (BN). At runtime, this BN is instantiated using real-time environmental observations to infer a probabilistic distribution over the safety estimation. This distribution enables the computation of both expected performance and its associated variance, providing a dynamic and context-aware measure of uncertainty. We demonstrate our approach through a case study of the Object Detection (OD) component in an Automated Valet Parking (AVP).