Adversarial Drift-Aware Predictive Transfer: Toward Durable Clinical AI

TL;DR

This paper introduces ADAPT, a framework that enhances the durability of clinical AI systems against temporal data shifts by optimizing worst-case performance with minimal retraining, thus maintaining reliability over time.

Contribution

ADAPT constructs an uncertainty set of models using historical and limited current data, enabling robust performance without extensive retraining or data sharing.

Findings

ADAPT outperforms baseline models in stability during coding transitions.

It significantly reduces annual performance decay in clinical predictions.

Validated on multiple healthcare datasets with consistent improvements.

Abstract

Clinical AI systems frequently suffer performance decay post-deployment due to temporal data shifts, such as evolving populations, diagnostic coding updates (e.g., ICD-9 to ICD-10), and systemic shocks like the COVID-19 pandemic. Addressing this ``aging'' effect via frequent retraining is often impractical due to computational costs and privacy constraints. To overcome these hurdles, we introduce Adversarial Drift-Aware Predictive Transfer (ADAPT), a novel framework designed to confer durability against temporal drift with minimal retraining. ADAPT innovatively constructs an uncertainty set of plausible future models by combining historical source models and limited current data. By optimizing worst-case performance over this set, it balances current accuracy with robustness against degradation due to future drifts. Crucially, ADAPT requires only summary-level model estimators from historical periods, preserving data privacy and ensuring operational simplicity. Validated on longitudinal suicide risk prediction using electronic health records from Mass General Brigham (2005--2021) and Duke University Health Systems, ADAPT demonstrated superior stability across coding transitions and pandemic-induced shifts. By minimizing annual performance decay without labeling or retraining future data, ADAPT offers a scalable pathway for sustaining reliable AI in high-stakes healthcare environments.