A causal model of safety assurance for machine learning

TL;DR

This paper introduces a causal model-based framework for safety assurance in machine learning, integrating safety engineering principles and structured evidence analysis to improve safety argumentation.

Contribution

It presents a novel causal modeling framework for ML safety assurance, including evidence categorization and structured analysis methods, advancing safety argumentation practices.

Findings

Framework effectively structures safety evidence for ML applications

Formalizations illustrate causal relationships and safety contributions

Proposes future research directions for safety assurance in ML

Abstract

This paper proposes a framework based on a causal model of safety upon which effective safety assurance cases for ML-based applications can be built. In doing so, we build upon established principles of safety engineering as well as previous work on structuring assurance arguments for ML. The paper defines four categories of safety case evidence and a structured analysis approach within which these evidences can be effectively combined. Where appropriate, abstract formalisations of these contributions are used to illustrate the causalities they evaluate, their contributions to the safety argument and desirable properties of the evidences. Based on the proposed framework, progress in this area is re-evaluated and a set of future research directions proposed in order for tangible progress in this field to be made.