Monitoring machine learning (ML)-based risk prediction algorithms in the presence of confounding medical interventions

TL;DR

This paper addresses the challenge of monitoring ML risk prediction models in healthcare when confounding medical interventions occur, proposing a new method that accounts for these confounders to ensure valid performance assessment.

Contribution

It introduces a novel score-based CUSUM monitoring procedure that accounts for confounding interventions and demonstrates its effectiveness through simulations and real-world data.

Findings

Monitoring is valid under certain assumptions despite confounding interventions.

Combining model updating with monitoring improves detection of performance issues.

The method successfully detects calibration decay in a postoperative nausea risk model.

Abstract

Performance monitoring of machine learning (ML)-based risk prediction models in healthcare is complicated by the issue of confounding medical interventions (CMI): when an algorithm predicts a patient to be at high risk for an adverse event, clinicians are more likely to administer prophylactic treatment and alter the very target that the algorithm aims to predict. A simple approach is to ignore CMI and monitor only the untreated patients, whose outcomes remain unaltered. In general, ignoring CMI may inflate Type I error because (i) untreated patients disproportionally represent those with low predicted risk and (ii) evolution in both the model and clinician trust in the model can induce complex dependencies that violate standard assumptions. Nevertheless, we show that valid inference is still possible if one monitors conditional performance and if either conditional exchangeability or time-constant selection bias hold. Specifically, we develop a new score-based cumulative sum (CUSUM) monitoring procedure with dynamic control limits. Through simulations, we demonstrate the benefits of combining model updating with monitoring and investigate how over-trust in a prediction model may delay detection of performance deterioration. Finally, we illustrate how these monitoring methods can be used to detect calibration decay of an ML-based risk calculator for postoperative nausea and vomiting during the COVID-19 pandemic.