Development and interpretable machine learning models for classification of pancreatic pseudocyst risk in acute pancreatitis

TL;DR

This study developed and validated a machine learning model to predict the risk of pancreatic pseudocysts in acute pancreatitis patients using clinical data.

Contribution

The study introduces an interpretable machine learning model for predicting pancreatic pseudocyst risk with temporal validation.

Findings

The random forest model achieved high AUC scores on both internal and temporal validation sets.

Serum calcium and C-reactive protein were identified as the most important predictors for pseudocyst classification.

The model uses seven routinely available clinical indicators for classification.

Abstract

Pancreatic pseudocysts (PPC) are a late local complication of acute pancreatitis (AP). Persistent PPC carry a high risk of severe outcomes. Existing models, which are predominantly based on logistic regression, exhibit limited predictive performance and have not undergone temporal validation. This study aimed to develop and validate an interpretable machine learning model using routinely available clinical data for classifying AP patients according to PPC development status.

We retrospectively analyzed 1,184 AP patients admitted to a tertiary hospital between 2018 and 2023. Data from 2018 to 2022 (n = 979) were randomly split into training (70%, n = 685) and internal test (30%, n = 294) sets, while the 2023 cohort (n = 205) served as an independent temporal validation set. Candidate predictors—including demographic characteristics, clinical history, and routine laboratory parameters—were screened via univariate analysis and further selected using LASSO regression to address multicollinearity. Nine machine learning algorithms were developed and compared: logistic regression, decision tree, random forest, artificial neural network, support vector machine, K-nearest neighbors, naïve Bayes, XGBoost, and LightGBM. Model performance was evaluated using area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value and negative predictive value. The optimal model was interpreted using SHapley Additive exPlanations (SHAP).

LASSO regression selected seven predictors: diabetes history, pancreatitis history, biliary surgery history, C-reactive protein, albumin, blood urea nitrogen, and serum calcium. The random forest model demonstrated the best classification performance, achieving an AUC of 0.884 (95% CI: 0.827–0.941) on the internal test set and 0.914 (95% CI: 0.845–0.983) on the temporal validation set. SHAP analysis identified serum calcium and C-reactive protein as the most important predictors, with low calcium and elevated CRP substantially increasing the probability of PPC classification.

We developed and temporally validated interpretable machine learning models for classifying PPC development status using seven routinely available clinical indicators. The random forest model demonstrated excellent discrimination and generalizability, while SHAP analysis provided transparent explanations of individual classifications. These models may facilitate early identification of high-risk AP patients and guide proactive clinical management.

Figures