Bootstrapping-based Regularisation for Reducing Individual Prediction Instability in Clinical Risk Prediction Models

TL;DR

This paper introduces a bootstrapping-based regularisation method for deep neural networks that enhances prediction stability and reproducibility in clinical risk models without compromising interpretability.

Contribution

The study presents a novel regularisation framework embedding bootstrapping into training, improving stability and reproducibility of clinical prediction models compared to conventional and ensemble methods.

Findings

Improved prediction stability with lower mean absolute differences.

High feature importance consistency across models.

Maintained discriminative performance despite increased stability.

Abstract

Clinical prediction models are increasingly used to support patient care, yet many deep learning-based approaches remain unstable, as their predictions can vary substantially when trained on different samples from the same population. Such instability undermines reliability and limits clinical adoption. In this study, we propose a novel bootstrapping-based regularisation framework that embeds the bootstrapping process directly into the training of deep neural networks. This approach constrains prediction variability across resampled datasets, producing a single model with inherent stability properties. We evaluated models constructed using the proposed regularisation approach against conventional and ensemble models using simulated data and three clinical datasets: GUSTO-I, Framingham, and SUPPORT. Across all datasets, our model exhibited improved prediction stability, with lower mean absolute differences (e.g., 0.019 vs. 0.059 in GUSTO-I; 0.057 vs. 0.088 in Framingham) and markedly fewer significantly deviating predictions. Importantly, discriminative performance and feature importance consistency were maintained, with high SHAP correlations between models (e.g., 0.894 for GUSTO-I; 0.965 for Framingham). While ensemble models achieved greater stability, we show that this came at the expense of interpretability, as each constituent model used predictors in different ways. By regularising predictions to align with bootstrapped distributions, our approach allows prediction models to be developed that achieve greater robustness and reproducibility without sacrificing interpretability. This method provides a practical route toward more reliable and clinically trustworthy deep learning models, particularly valuable in data-limited healthcare settings.