Is Your Model Risk ALARP? Evaluating Prospective Safety-Critical Applications of Complex Models

TL;DR

This paper discusses evaluating the safety risk of complex models in safety-critical industries, proposing methods to ensure model risk is minimized to an acceptable level while leveraging their benefits.

Contribution

It introduces a framework combining decision analysis, uncertainty quantification, and value of information to assess when model risk is ALARP in safety-critical applications.

Findings

Demonstrates automated weld radiograph classification example

Proposes a method to evaluate if model risk is ALARP

Integrates statistical decision analysis with uncertainty quantification

Abstract

The increasing availability of advanced computational modelling offers new opportunities to improve safety, efficacy, and emissions reductions. Application of complex models to support engineering decisions has been slow in comparison to other sectors, reflecting the higher consequence of unsafe applications. Adopting a complex model introduces a \emph{model risk}, namely the expected consequence of incorrect or otherwise unhelpful outputs. This should be weighed against the prospective benefits that the more sophisticated model can provide, also accounting for the non-zero risk of existing practice. Demonstrating when the model risk of a proposed machine learning application is As Low As Reasonably Practicable (ALARP) can help ensure that safety-critical industries benefit from complex models where appropriate while avoiding their misuse. An example of automated weld radiograph classification is presented to demonstrate how this can be achieved by combining statistical decision analysis, uncertainty quantification, and value of information.