Robust by Design: A Continuous Monitoring and Data Integration Framework for Medical AI

TL;DR

This paper introduces a continuous monitoring and data integration framework for medical AI that maintains high performance amid data drift by selectively retraining with similar, low-uncertainty images.

Contribution

It presents a novel three-stage method combining multi-metric analysis and uncertainty gating for autonomous model updating in clinical environments.

Findings

Framework prevents significant performance degradation (~0.92 AUC, ~89% accuracy)

Selective data integration avoids catastrophic forgetting

Effective in multi-center medical imaging dataset

Abstract

Adaptive medical AI models often face performance drops in dynamic clinical environments due to data drift. We propose an autonomous continuous monitoring and data integration framework that maintains robust performance over time. Focusing on glomerular pathology image classification (proliferative vs. non-proliferative lupus nephritis), our three-stage method uses multi-metric feature analysis and Monte Carlo dropout-based uncertainty gating to decide when to retrain on new data. Only images statistically similar to the training distribution (via Euclidean, cosine, Mahalanobis metrics) and with low predictive entropy are integrated. The model is then incrementally retrained with these images under strict performance safeguards (no metric degradation >5%). In experiments with a ResNet18 ensemble on a multi-center dataset, the framework prevents performance degradation: new images were added without significant change in AUC (~0.92) or accuracy (~89%). This approach addresses data shift and avoids catastrophic forgetting, enabling sustained learning in medical imaging AI.