Impact of blood culture positivity at intensive care unit admission on mortality in infective endocarditis: Machine learning and deep learning-based causal inference models

TL;DR

Blood culture positivity at ICU admission is linked to higher mortality in infective endocarditis patients, as shown using machine learning and causal inference models.

Contribution

Novel use of machine learning and causal inference models to assess the impact of blood culture positivity on mortality in ICU infective endocarditis patients.

Findings

Blood culture positivity was a top predictor of mortality in ICU infective endocarditis patients.

Machine learning models showed blood culture positivity increased mortality by up to 7.4% in test data.

The effect was strongest in older patients and those with low blood pressure.

Abstract

Infective endocarditis (IE) carries high in-hospital mortality, particularly among intensive care unit (ICU) patients. The predictive role of blood culture positivity in these patients remains unclear.

We analyzed 484 adult IE patients from the Medical Information Mart for Intensive Care III (MIMIC-III) database, divided into training (n = 339) and testing (n = 145) cohorts. A suite of demographic, clinical, laboratory, and blood culture variables was used to develop tree-based machine learning models. Random Forest (RF) and Extreme Gradient Boosting (XGB) emerged as top performers and were combined into an ensemble model. SHapley Additive exPlanations (SHAP) quantified variable importance, while the Generative Adversarial Nets for Inference of Individualized Treatment Effects (GANITE) model assessed the average treatment effect (ATE) and conditional treatment effects (CATE) of blood culture positivity on in-hospital mortality across various clinical subgroups.

The ensemble model demonstrated robust performance with an area under the receiver operating characteristic curve (AUROC) of 0.826 and an accuracy of 0.821 on the test set. Blood culture positivity consistently ranked among the top predictors of mortality. SHAP analysis revealed that the presence of bacteremia increased the predicted probability of in-hospital mortality. Specifically, the GANITE model estimated that blood culture positivity raised mortality by 0.9% (95% confidence interval [CI]: −0.9% to 2.6%) in the training set, 7.4% (95% CI: 4.3% to 10.4%) in the test set, and 2.8% (95% CI: 1.2% to 4.4%) overall. Furthermore, CATE analysis highlighted that the adverse impact of blood culture positivity was significantly more pronounced in patients aged 60 years and older, those with systolic blood pressure below 100 mmHg, and in certain endocarditis subtypes.

Blood culture positivity at ICU admission is associated with a modest yet clinically significant increase in in-hospital mortality among IE patients. The application of advanced machine learning and causal inference models enhances risk stratification and may inform more targeted clinical interventions in this high-risk group.

Figures

Study population.

Variable importance of Random Forest and Extreme Gradient Boosting models.Cr, creatinine; WBC, white blood count; PLT, platelet; HR, heart rate; Hb, hemoglobin; SBP, systolic blood pressure; Hct, hematocrit; K, potassium; DBP, diastolic blood pressure; SpO2, peripheral oxygen saturation; Na, sodium; MRSA, methicillin-resistant Staphylococcus aureus; NOS, not otherwise specified; MSSA, methicillin-sensitive Staphylococcus aureus.

Shapley additive explanations summary and waterfall plot of Random Forest and Extreme Gradient Boosting models.WBC, white blood cell; Cr, creatinine; PLT, platelet; SBP, systolic blood pressure; SpO2, peripheral oxygen saturation; HR, heart rate; Hb, hemoglobin; Hct, hematocrit; K, potassium; Na, sodium; MRSA, methicillin-resistant Staphylococcus aureus; DBP, diastolic blood pressure.

Bar plots of conditional average treatment effect.HR, heart rate; SBP, systolic blood pressure.

Full text

Introduction

The in-hospital mortality rate of infective endocarditis (IE) remains high, ranging from 13% to 25% [1–5]. IE can result in numerous complications requiring intensive care unit (ICU) care, such as septic shock and cardiogenic shock [6]. Mortality strikingly rises in patients requiring ICU admission, with rates escalating up to 24% to 54% [7]. Therefore, identifying the risk factors for mortality in IE patients, especially those admitted to the ICU, is critically important.

Known risk factors for mortality in IE patients include advanced age, diabetes mellitus, Staphylococcus aureus etiology, embolic events, clinical evolution with heart failure, and septic shock [8,9]. Specifically, for IE patients admitted to the ICU, risk factors for mortality include advanced age, stroke, high Simplified Acute Physiology Score II, Sequential Organ Failure Assessment scores, and Methicillin-resistant Staphylococcus aureus (MRSA) etiology [10,11]. The association between blood culture positivity and mortality in IE patients has been extensively studied, but there is some debate over whether blood culture positivity is a definitive risk factor for mortality [9,12,13]. In infective endocarditis, contemporary registry data (EURO-ENDO) report higher long-term mortality in culture-negative cases compared with culture-positive disease [14]. By contrast, several cohort studies—including a cardiac surgery series and community-acquired IE—reported no significant difference in short- or long-term mortality between culture-negative and culture-positive disease [15,16]. Consistently, a recent systematic review and meta-analysis found no overall mortality difference between culture-positive and culture-negative IE [17]. Taken together, these mixed findings motivate a cautious interpretation of culture positivity as a risk marker and justify our focus on absolute risk differences and sensitivity analyses. However, research on the relationship between blood culture positivity at the time of ICU admission and mortality in ICU-admitted IE patients is limited.

Thus, this study aimed to develop machine learning and deep learning models using the Medical Information Mart for Intensive Care III (MIMIC-III) dataset to predict in-hospital mortality among ICU-admitted endocarditis patients. We attempted to identify the risk factors and important variables in mortality prediction. Additionally, this study adopted a causal inference deep learning model to quantify the increment or decrement of in-hospital mortality based on blood culture positivity at the time of ICU admission.

Methods

Ethics statement

This retrospective study using the de-identified MIMIC-III database was approved by the Institutional Review Board of Seoul National University Hospital (IRB No. 2405-061-1535) with a waiver of informed consent.

Study population

In this study, we utilized the MIMIC-III dataset from Physionet and selected patients with a diagnosis related to endocarditis upon admission from those with ICU admission records. The MIMIC-III database contains deidentified health-related data from over forty thousand patients who stayed in critical care units at the Beth Israel Deaconess Medical Center between 2001 and 2012, offering a comprehensive repository of demographic, clinical, and outcome data [18].

The diagnoses related to endocarditis included a range of conditions such as Syphilitic endocarditis (Syphilitic endocarditis NOS, Syphilitic endocarditis of valve unspecified, Syphilitic endocarditis of mitral valve, Syphilitic endocarditis of aortic valve, Syphilitic endocarditis of tricuspid valve, Syphilitic endocarditis of pulmonary valve), Gonococcal endocarditis, Meningococcal endocarditis, Candida endocarditis, Histoplasmosis endocarditis (Infection by Histoplasma capsulatum endocarditis, Infection by Histoplasma duboisii endocarditis, Histoplasmosis unspecified endocarditis), Coxsackie endocarditis, acute and subacute bacterial endocarditis, acute and subacute infective endocarditis in disease classified elsewhere, acute endocarditis unspecified, endocarditis valve unspecified unspecified cause, endocarditis in disease classified elsewhere, other endocarditis valve unspecified, acute rheumatic endocarditis, and rheumatic disease of the endocardium valve unspecified. We included rheumatic endocarditis and rheumatic disease in this analysis, considering that infective endocarditis can coexist with rheumatic heart disease and that infective endocarditis can sometimes be misdiagnosed as rheumatic endocarditis [19,20]. Only adult patients were included in the analysis, and those with missing vital sign or laboratory data were excluded. For machine learning and deep learning analyses, we randomly divided the entire population into training and testing datasets at a ratio of 7:3.

Variables for analysis

Demographic data included age and sex. Initial vital signs comprised systolic blood pressure (SBP), diastolic blood pressure (DBP), heart rate (HR), and oxygen saturation (SpO_2_). Laboratory data included white blood cell count (WBC), hemoglobin (Hb), hematocrit (Hct), platelet count (PLT), initial creatinine (initial Cr), baseline creatinine (base Cr), bicarbonate, sodium, and potassium. Laboratory values were taken from the time 24 hours before ICU admission up to 6 hours after admission, with the value measured closest to admission time utilized. Blood culture data included blood culture positivity, Methicillin-sensitive Staphylococcus aureus (MSSA) bacteremia, MRSA bacteremia, Pseudomonas bacteremia, and candidemia. Since it takes at least 48 hours to confirm blood culture positivity [21,22], we collected results from tests performed 48 hours before ICU admission up to 6 hours after admission and used the result closest to the ICU admission time. Additionally, surgery codes were used to determine the presence of open heart surgery, septal repair, annuloplasty, or other heart surgeries. Other vital signs and the norepinephrine rate measured closest to ICU admission time were also included. The occurrence of intubation within 6 hours of admission was also considered as a variable.

For comparing variables between groups with and without in-hospital mortality, t-tests were used for continuous variables, and chi-square tests were used for categorical variables. Multivariable logistic regression analysis was conducted using all variables to assess the association between in-hospital mortality and each variable. A P value <0.05 was considered statistically significant.

Machine learning and deep learning models predicting in-hospital mortality

The analysis was conducted using the Python PyCaret package, where ten different tree-based algorithms were applied to the training data with 10-fold cross-validation: Extra Trees, CatBoost, Extreme Gradient Boosting (XGB), Random Forest (RF), Light Gradient Boosting Machine, Gradient Boosting, Decision Tree, Ada Boost, Logistic Regression, Ridge, Linear Discriminant Analysis, and Support Vector Machine. During training, we performed 10-fold cross-validation on the training data to select the top two algorithms with the highest average area under the receiver operating characteristic curve (AUROC) values. These two algorithms then underwent hyperparameter tuning with another 10-fold cross-validation. Subsequently, the variable importance for in-hospital mortality prediction was assessed for these two algorithms. The predictive performance of these models was then evaluated on the test data. Additionally, SHapley Additive exPlanations (SHAP) values were calculated for the entire dataset using the top two algorithms, creating summary plots to illustrate the impact of each variable on in-hospital mortality. SHAP values, based on game theory, help to demystify machine learning predictions by quantifying the contribution of each input variable [23].

Evaluation metrics for model performance included accuracy, AUROC, and F1-score. The performance of the models was compared using receiver operating characteristic (ROC) curves and decision curve analysis (DCA) plots. For deep learning, the TabTransformer model was utilized. After training on the training data using the same methodology as in machine learning, model performance was evaluated on the test data.

Causal inference deep learning model

We used generative adversarial nets for inference of individualized treatment effects (GANITE) as an established causal deep learning framework that (i) imputes counterfactual outcomes via an adversarial generator–discriminator and (ii) then estimates individualized effects with an inference network [24]. We chose GANITE not for methodological novelty but because it is well validated, flexibly accommodates non-linear covariate–treatment interactions and treatment-effect heterogeneity, and yields patient-level probabilistic potential outcomes that map directly to absolute risk differences—a clinically interpretable quantity for our binary mortality endpoint. Compared with representation-balancing or propensity-regularized alternatives, GANITE’s direct counterfactual modeling could align better with our goal of estimating admission-time risk differences under routine ICU data, while maintaining transparency and stability for sensitivity analyses. Blood culture positivity was set as the treatment variable, while in-hospital mortality was designated as the outcome variable. To assess the effectiveness of the model’s training, accuracy, AUROC, and F1-score were evaluated on the test dataset. Additionally, calibration performance was analyzed using calibration plots for the training and test data.

Furthermore, the average treatment effect (ATE) was calculated to determine how mortality changed, on average, when bacteremia was positive compared to when it was negative across the training, test, and total datasets. Conditional treatment effects (CATE) were also assessed based on age (60 years and older vs. younger than 60), intubation within the first 6 hours, the presence of open heart surgery, sex, heart rate (≥100 vs. < 100 beats per minute), SBP (≥100 vs. < 100 mmHg), norepinephrine administration, and endocarditis etiology (bacterial, rheumatic, and candida), as well as the performance of annuloplasty. T-tests were employed to compare CATE between different groups.

To delve deeper into the impact of blood culture positivity on in-hospital mortality, the dataset was split into two groups based on the extent to which blood culture positivity influenced the likelihood of in-hospital mortality: whether the influence of blood culture positivity on the probability of in-hospital mortality was positive or negative. The variables used in the analysis were compared between these two groups. Differences in continuous variables between these groups were analyzed using t-tests, while categorical variables were compared using chi-square tests. Furthermore, a multivariable logistic regression analysis was conducted to examine the odds ratio for whether blood culture positivity increases the probability of in-hospital mortality, thereby identifying which variable characteristics exert a more pronounced effect on the likelihood of blood culture positivity.

Sensitivity analysis

We performed two prespecified sensitivity analyses. First, to assess generalizability and address potential overfitting, we conducted stratified 5-fold cross-validation (stratified by in-hospital mortality). In each outer fold, the GANITE model was trained on four folds with all hyperparameters fixed to those of the primary analysis, and evaluated on the held-out fold. We also formed out-of-fold (OOF) predictions by concatenating all held-out predictions across folds to report pooled AUROC and pooled ATE. Pooled ATE 95% confidence intervals (CIs) were obtained via nonparametric bootstrap with 500 replicates. Second, because rheumatic endocarditis could confound inference, we excluded patients with rheumatic endocarditis and repeated the same 5-fold cross-validation protocol, using identical covariates, preprocessing, and hyperparameters as in the primary analysis.

Results

Study population

Out of a total of 519 endocarditis patients, 484 patients with complete vital sign and laboratory data were included in the analysis, as illustrated in Fig 1. A total of 484 patients were included in the analysis, among which 71 (14.7%) experienced in-hospital mortality. The mean age of the study population was 59.3 years, with the age of patients who died due to endocarditis being significantly higher (mean age 65.1 years) compared to those who survived (mean age 58.2 years) (p = 0.001). The mortality group had significantly lower percentages of males and those who had open heart surgery, but a higher incidence of bacteremia (Table 1). The dataset was randomly divided into 339 training and 145 testing datasets, with no significant differences in nearly all variables between the two groups (S1 Table).

Table 1: Baseline characteristics.

Study population.

Logistic regression for in-hospital mortality

After adjusting for all variables, the multivariable logistic regression analysis showed that higher age (Odds Ratio [OR]: 1.03 [1.01–1.05], p: 0.010), initial creatinine (OR: 1.70 [1.08–2.69], p: 0.220), and WBC (OR: 1.10 [1.05–1.14], p: < 0.001) were associated with an increased risk of in-hospital mortality. Conversely, lower bicarbonate levels (OR: 0.90 [0.84–0.96], p: 0.001) were linked to a higher mortality risk. Blood culture positive, MRSA bacteremia, and MSSA bacteremia did not show a statistically significant correlation with in-hospital mortality (Table 2).

Table 2: Odds ratio for in-hospital mortality.

Machine learning and deep learning models predicting in-hospital mortality

In the 10-fold cross-validation on the training data, the RF and XGB models demonstrated the highest performance. Following hyperparameter tuning, an ensemble model combining these two was developed. The RF, XGB, and ensemble models exhibited AUROCs of 0.821, 0.809, and 0.826, respectively, and accuracies of 0.814, 0.807, and 0.821 in the test data (S2 Table and S1 Fig). The TabTransformer model showed an AUROC of 0.645 and an accuracy of 0.841 on the test data. Decision Curve Analysis (DCA) plots indicated that the RF, XGB, and ensemble models performed similarly well, whereas the TabTransformer showed relatively poorer performance.

The variable importance analysis in the RF and XGB models identified blood culture positivity as the fourth and fifth most critical variable for predicting in-hospital mortality, respectively (Fig 2). The SHAP value plots indicated a tendency for both models to predict in-hospital mortality when bacteremia was positive (Fig 3). Additionally, trends suggested that lower heart rate, lower WBC counts, the performance of open heart surgery, younger age, and lower norepinephrine administration rates were associated with lower risks of in-hospital mortality.

Variable importance of Random Forest and Extreme Gradient Boosting models.Cr, creatinine; WBC, white blood count; PLT, platelet; HR, heart rate; Hb, hemoglobin; SBP, systolic blood pressure; Hct, hematocrit; K, potassium; DBP, diastolic blood pressure; SpO2, peripheral oxygen saturation; Na, sodium; MRSA, methicillin-resistant Staphylococcus aureus; NOS, not otherwise specified; MSSA, methicillin-sensitive Staphylococcus aureus.

Shapley additive explanations summary and waterfall plot of Random Forest and Extreme Gradient Boosting models.WBC, white blood cell; Cr, creatinine; PLT, platelet; SBP, systolic blood pressure; SpO2, peripheral oxygen saturation; HR, heart rate; Hb, hemoglobin; Hct, hematocrit; K, potassium; Na, sodium; MRSA, methicillin-resistant Staphylococcus aureus; DBP, diastolic blood pressure.

Causal inference deep learning model

The GANITE model analysis revealed that blood culture positive cases at ICU admission increased the probability of in-hospital mortality by 0.9% (95% [CI]: −0.9–2.6%) in the training data, 7.4% (95% CI: 4.3–10.4%) in the test data, and 2.8% (95% CI: 1.2–4.4%) across the total data (Table 3). The GANITE model’s AUROC was 0.768, with an accuracy of 0.834 in the test data (S2 Fig). In the test data, the GANITE model showed relatively moderate calibration performance but tended to slightly overestimate when predicting high in-hospital mortality.

Table 3: Average treatment effect for in-hospital mortality and evaluation indexes of deep learning based causal inference model.

The CATE analysis showed that blood culture positivity increased the tendency for in-hospital mortality under all conditions except in subjects with rheumatic endocarditis (Fig 4). However, the 95% CI for the heart rate > 100, norepinephrine, and annuloplasty groups included zero, indicating no significant effect. Statistically significant differences were observed when comparing the CATEs of opposing conditions in males, subjects with rheumatic endocarditis, and Candida endocarditis. Other conditions showed no statistically significant differences.

Bar plots of conditional average treatment effect.HR, heart rate; SBP, systolic blood pressure.

The median effect of blood culture positivity on in-hospital mortality was 1.57%. When dividing the groups into those where the effect of blood culture positivity was ≥ 1.57% or not, no significant differences were found in variables other than age, SBP, DBP, bicarbonate, creatinine, baseline creatinine, platelet, and potassium, including open heart surgery, annuloplasty, heart rate, WBC count, and norepinephrine administration rate (S3 Table). Only age, SBP, and PLT showed significant correlations in multivariable logistic regression for high blood culture positivity’s impact on in-hospital mortality, which is over 1.57% (Table 4). Older age, lower SBP, and higher PLT were associated with a higher effect of blood culture positivity on in-hospital mortality.

Table 4: Odds ratio for high effectiveness of blood culture positive for in-hospital mortality.

Sensitivity analysis

In 5-fold cross-validation, the mean AUROC across folds was 0.670 ± 0.047, with a pooled OOF AUROC of 0.666. The mean ATE across folds was 4.2% ± 3.0%, and the pooled OOF ATE was 4.2% (95% CI, 2.7%−5.7%) (S4 Table). After excluding rheumatic endocarditis and repeating the analysis, the mean AUROC was 0.709 ± 0.066, with a pooled OOF AUROC of 0.697. The mean ATE across folds was 4.6% ± 1.9%, and the pooled OOF ATE was 4.6% (95% CI, 3.4%−5.8%) (S5 Table).

Discussion

This study not only developed a machine learning model to predict in-hospital mortality among ICU-admitted endocarditis patients but also explored which variables influence mortality through statistical and machine learning techniques. Specifically, by employing a deep learning causal inference model for the first time, we observed an association between blood culture positivity and a slightly higher in-hospital mortality under similar patient conditions. Importantly, these patterns remained robust in prespecified sensitivity analyses, including patient-level stratified 5-fold cross-validation and a cohort restricted to non-rheumatic endocarditis. Our results showed that the ensemble model achieved the highest AUROC of 0.821 in the test dataset, suggesting that the mortality prediction model itself could aid clinical practice.

We investigated the impact of various factors on in-hospital mortality using logistic regression—a classical statistical model well-known for its ability to elucidate the associations between clinical variables and outcomes. However, logistic regression has limitations, such as assessing the model’s goodness of fit. Thus, we supplemented this with an analysis of SHAP values from tree-based machine learning models, offering additional clinical insights into the relationship between variables and mortality. Both logistic regression and machine learning models indicated that patients not undergoing open heart surgery tended to have a lower risk of in-hospital mortality. This observation aligns with previous studies [3,9].

Although blood culture positivity did not show significant results in logistic regression, machine learning models revealed its tendency to influence mortality. This discrepancy led us to employ a deep learning causal inference model to assess the association and estimate potential effects of bacteremia on in-hospital mortality. Since randomization is not possible for the presence of blood culture positivity, using a deep learning causal inference model to explore potential causal relationships under standard assumptions is very useful in evaluating the impact of bacteremia on clinical outcomes. In this study, we leveraged the causal inference deep learning framework to elucidate the impact of blood culture positivity on mortality rates among ICU patients with endocarditis. Unlike traditional statistical methods, which often struggle with direct assessments and may overestimate effects in certain sample sizes, the causal inference deep learning model provides numerical estimates of how blood culture positivity may be associated with mortality risk. This nuanced analysis yields a more intuitive interpretation of results, differentiating it markedly from classical statistical approaches [25]. Beyond merely conducting subgroup analyses or calculating CATE, the causal inference deep learning model adeptly segregates patient populations based on their predicted mortality risk depending on blood culture positivity. It identifies characteristics of patients for whom blood culture positivity may be associated with a higher mortality risk, thereby offering clearer insights into patient vulnerability.

The presence of persistent blood culture positivity, particularly blood culture positivity at ICU admission, influencing patient survival outcomes has yet to be definitively established. Whether blood culture positivity is a definitive risk factor for mortality remains a subject of debate [13]. However, there is previous research indicating that persistent bacteremia serves as a risk factor for mortality [26]. The presence of blood culture positivity at the time of ICU admission suggests a high bloodstream pathogen burden and a potential state of severe sepsis. For this reason, the presence of blood culture positivity at ICU admission could be thought to increase the risk of mortality. However, it is challenging to assume that blood culture-negative endocarditis has a lower mortality risk due to a lower bloodstream pathogen burden, given the heterogeneous nature of bacteremia-negative endocarditis patients, including those with rheumatic endocarditis [27]. Furthermore, some endocarditis cases may be caused by specific pathogens not detectable by blood culture, leading to variations in mortality among blood culture-negative endocarditis depending on the etiology [27]. Therefore, our study results suggest that blood culture positivity was associated with a higher mortality risk, the magnitude of this increase was not substantial.

Old age and lower SBP subjects appeared to have higher effects of blood culture positivity on in-hospital mortality. The mortality due to severe sepsis in elderly patients is 1.3–1.5 times higher than that in younger cohorts [28,29]. Therefore, blood culture positivity at ICU admission in older patients is likely to predict higher mortality. The tendency for lower SBP to be associated with higher mortality when blood culture is positive may reflect the high mortality observed in septic shock scenarios where blood pressure drops and blood cultures are positive [30].

Several limitations are acknowledged in this research. First, although the total number of subjects was 484—which is substantial for an ICU endocarditis cohort—it remains modest for machine and deep learning analyses. In addition, extreme sparsity in several subgroups limits statistical power for both model training and heterogeneity analyses, increasing variance, heightening the risk of overfitting, and reducing generalizability. Obtaining large-scale data under specific conditions like ICU admissions for endocarditis is challenging. Second, the absence of variables related to heart function, such as vegetation size and ejection fraction, due to the constraints of the MIMIC-III dataset, prevented their inclusion. Incorporating these variables would have significantly reduced the analysis pool due to missing data. Third, we observed a discrepancy in ATE CIs between the development-split training and testing sets, likely reflecting overfitting and sampling variability in a modest-sized cohort. Our out-of-fold, patient-level stratified 5-fold cross-validation mitigated this issue and yielded more stable—albeit modest—effects; nevertheless, the small magnitudes warrant cautious interpretation and external validation. Fourth, the overall cohort size and extreme sparsity in several subgroups limit statistical power for both model training and heterogeneity analyses, increasing variance and the risk of overfitting and reducing generalizability. Fourth, key infective endocarditis prognostic variables—most notably echocardiographic features such as vegetation size, ejection fraction, and valve involvement—were unavailable in MIMIC-III. As a result, residual confounding may persist despite adjustment for available clinical.

In conclusion, our study suggests that the presence of blood culture positivity at ICU admission in ICU-admitted endocarditis patients was associated with a small increase in-hospital mortality compared to its absence. This finding, supported by advanced machine learning and causal inference analyses, highlights specific patient characteristics associated with higher estimated risk when blood culture positivity is present.

Supporting information

S1 TableBaseline characteristics of train and test sets.(DOCX)

S2 TableModel performance for in-hospital mortality prediction.(DOCX)

S3 TableComparison by treatment-effect strata.(DOCX)

S4 TableStratified 5-fold cross-validation results.(DOCX)

S5 TableCross-validation excluding rheumatic endocarditis.(DOCX)

S1 FigROC and decision curve analysis of prediction models.(DOCX)

S2 FigROC and calibration of the causal inference model.(DOCX)