Causal Falsification of Digital Twins

TL;DR

This paper introduces a causal inference-based method to rigorously assess the accuracy of digital twins using observational data, avoiding unconfoundedness assumptions, and demonstrates its application in a healthcare case study.

Contribution

It proposes a novel statistical procedure to falsify digital twins' correctness without relying on unconfounded data assumptions, enhancing safety in critical applications.

Findings

Method reliably identifies incorrect digital twins.

Applicable to confounded observational datasets.

Validated on sepsis modeling case study.

Abstract

Digital twins are virtual systems designed to predict how a real-world process will evolve in response to interventions. This modelling paradigm holds substantial promise in many applications, but rigorous procedures for assessing their accuracy are essential for safety-critical settings. We consider how to assess the accuracy of a digital twin using real-world data. We formulate this as causal inference problem, which leads to a precise definition of what it means for a twin to be "correct" appropriate for many applications. Unfortunately, fundamental results from causal inference mean observational data cannot be used to certify that a twin is correct in this sense unless potentially tenuous assumptions are made, such as that the data are unconfounded. To avoid these assumptions, we propose instead to find situations in which the twin is not correct, and present a general-purpose statistical procedure for doing so. Our approach yields reliable and actionable information about the twin under only the assumption of an i.i.d. dataset of observational trajectories, and remains sound even if the data are confounded. We apply our methodology to a large-scale, real-world case study involving sepsis modelling within the Pulse Physiology Engine, which we assess using the MIMIC-III dataset of ICU patients.