Comparison of the Prognostic Performance of Various Machine Learning Models in Patients with Acute Myocardial Infarction: Results from the COREA-AMI Registry

TL;DR

This study compares machine learning models to predict heart-related risks in patients who had heart attacks and found that the random forest model performed best.

Contribution

The study evaluates and compares various machine learning models for predicting long-term outcomes in acute myocardial infarction patients who underwent PCI.

Findings

The random forest model outperformed other ML techniques and logistic regression in predicting MACEs.

Key predictors of MACEs included age, renal function, and adherence to medical therapies.

The RF model achieved an AUC of 0.822, accuracy of 0.804, and F1 score of 0.870 at 5 years.

Abstract

Background and Objectives: To date, several machine learning (ML) prognostic prediction models have been investigated for patients with acute myocardial infarction (AMI). However, few studies have compared the prognostic performance of ML techniques in AMI patients who underwent percutaneous coronary intervention (PCI). We sought to compare the prognostic performance among various machine learning techniques to determine which one showed the best prediction ability. Materials and Methods: Using data from the large, multicenter COREA-AMI registry, this study analyzed 10,172 patients to predict major adverse cardiac events (MACEs) at 1 and 5 years. MACE was defined as a composite of cardiac death, myocardial infarction, or cerebrovascular accident. Results: Compared with the four other ML techniques and traditional logistic regression, the random forest (RF) model consistently demonstrated the highest predictive performance. At 5 years, the RF model achieved a superior area under the curve (AUC) of 0.822, an accuracy of 0.804, and an F1 score of 0.870. To ensure clinical interpretability, a SHapley Additive exPlanations analysis was performed on the RF model. It identified key independent predictors for MACEs. The top nonmodifiable predictors included age, renal function, and left ventricular ejection fraction, whereas modifiable risk factors included dual antiplatelet therapy, statin therapy, angiotensin-converting enzyme inhibitor/angiotensin receptor blocker therapy, and adherence to these optimal medical therapy. Conclusions: In this real-world patient cohort, the RF model provided modest improvements in long-term risk stratification, and our findings highlight the continuing importance of guideline-directed medical therapy in determining patient prognosis.

Figures

Full text

1. Introduction

In Korea, the prevalence of acute myocardial infarction (AMI) has increased over the past 10 years [1]. AMI has consistently been associated with poorer clinical outcomes compared with a first event [2,3]. Therefore, secondary prevention with guideline-derived optimal medical therapy is important for reducing the risk of recurrent adverse cardiovascular events [4].

Acute myocardial infarction (AMI) remains a leading cause of death globally [5,6], making accurate prognostic evaluation essential for risk stratification and the selection of optimal treatment strategies for patients. Historically, traditional statistical methods such as logistic regression analysis have served as the cornerstone for evaluating prognostic predictors in this population. However, these models are often limited in their ability to capture complex, nonlinear relationships. In recent years, machine learning (ML) techniques have emerged as powerful tools for developing predictive models capable of identifying complex and nonlinear patterns within large datasets of patients with coronary artery disease [7]. Several studies have demonstrated the clinical utility of ML in predicting in-hospital and midterm outcomes after AMI [8,9].

However, data remain limited regarding the long-term comparative performance of different ML algorithms, especially in large real-world cohorts of AMI patients treated with percutaneous coronary intervention (PCI). Furthermore, a significant barrier to the clinical adoption of many ML models is their ‘black box’ nature. For clinicians to trust and act upon these predictions, it is crucial that the models be not only accurate but also interpretable.

Therefore, we aimed to compare the long-term prognostic performance of several ML techniques, including traditional logistic regression, and to identify major clinical predictors using interpretable ML-based approaches.

2. Materials and Methods

2.1. Study Design and Population

The CardiOvascular Risk and identification of potential high-risk populations in the Acute Myocardial Infarction (COREA-AMI) registry is a retrospectively collected, prospective, multicenter, all-comer cohort study designed to evaluate clinical outcomes and risk factors in patients with AMI. The registry was established and is primarily coordinated by the Catholic University of Korea and its affiliated hospitals in collaboration with multiple tertiary referral centers across the nation under the support of the Korean Society of Interventional Cardiology. A total of 10,719 patients who underwent PCI between April 2004 and April 2015 were enrolled in the registry.

Among the 10,719 patients, we included consecutive patients who were treated with drug-eluting stents (DESs) during the index PCI for the current analysis. Patients who did not undergo stent implantation or who were lost to follow-up were excluded (Figure 1). To reduce potential multicollinearity and ensure model robustness, we excluded redundant variables that conveyed overlapping information (e.g., body mass index [BMI] instead of height and weight and age instead of date of birth). In addition, variables of in-hospital adverse events (such as cardiogenic shock, use of inotropes, dialysis, transfusion, and acute heart failure) were not included, as they represent in-hospital outcomes rather than predictors. Finally, a total of 64 variables were selected for analysis (Table 1), including 7 demographic variables, 14 related to past medical history, 14 laboratory and echocardiographic variables, 7 discharge medication variables, and 22 procedural variables.

2.2. Ethics Statement

The study protocol of the COREA-AMI registry was conducted in accordance with the Declaration of Helsinki and was approved by the Institutional Review Boards (IRBs) of all the participating centers (IRB approval code: XC13RIMI0060K) on 25 July 2013. It was also registered at ClinicalTrials.gov (NCT02385682). Written informed consent was not needed, as this study was approved by the Institutional Review Boards in accordance with national regulations for retrospective observational studies.

2.3. Study Outcomes

The primary endpoint was the rate of major adverse cardiac events (MACEs) at 5 years, defined as a composite of cardiac death, MI and cerebrovascular accident. The secondary endpoint was the occurrence of MACEs at 1 year. All clinical events were recorded at each participating hospital and were centrally adjudicated by an independent panel of clinicians. To assess long-term prognostic performance, we selected 5-year MACEs as the primary endpoint. In addition, we analyzed 1-year MACEs to examine differences in model performance and predictor contributions between the 1-year and 5-year outcomes. This approach also enabled us to evaluate how the relevance of specific predictors evolves over time.

2.4. Statistical Method and Machine Learning Model

Continuous variables are presented as the mean values with standard deviations, and categorical variables are presented as frequencies with percentages. The registry originally included 121 variables, including past disease history, laboratory findings, procedural variables, medications and demographic characteristics. As described in Section 2.1, after excluding redundant variables and those reflecting in-hospital events, a total of 64 variables were finalized for model construction and subsequent analysis.

Missing data were assumed to be Missing at Random (MAR) and were addressed using multiple imputation based on the Markov Chain Monte Carlo (MCMC) method implemented through the PROC MI procedure in SAS (version 9.4), generating five imputed datasets under the assumption of multivariate normality [10,11,12]. The proportion of missing values varied across variables, ranging from 0.3% to 38.4%, with HbA1C showing the highest rate (38.4%) and most variables having less than 10% missing values. Missingness was observed in body mass index (BMI, 3.9%), systolic blood pressure (SBP, 1.1%), diastolic blood pressure (DBP, 1.2%), heart rate (HR, 1.2%), left ventricular ejection fraction (LVEF, 7.0%), white blood cell (WBC, 0.5%), neutrophil (0.5%), hemoglobin (Hb, 0.3%), platelet (0.4%), glycated Hb (HbA1c, 38.4%), total cholesterol (TC, 7.1%), triglyceride (TG, 7.6%), high-density lipoprotein cholesterol (HDL-C, 7.7%), low-density lipoprotein cholesterol (LDL-C, 10.1%), high-sensitivity C-reactive protein (hsCRP, 16.5%), creatinine (Cr, 0.3%), estimated glomerular filtration rate (eGFR, 0.3%), and creatine kinase–myocardial band (CK-MB, 0.7%), whereas all other variables had no missing values.

The primary outcome, MACE, was defined as a binary variable (0 = no event, 1 = event). Given the binary nature of the outcome and the potential for class imbalance, the dataset was split into training (70%) and testing (30%) subsets using stratified sampling to maintain the proportion of MACE cases [13,14]. To ensure robustness against sampling bias and random variability, the entire process of data splitting, model training with hyperparameter tuning, and evaluation was repeated 100 times, and the average performance metrics across these iterations were used for final comparison [15].

Prediction models were developed and evaluated using five machine learning algorithms: traditional logistic regression, naïve Bayes (NB), elastic net regression, support vector machine (SVM), and random forest (RF) [16,17,18,19,20]. For each model, hyperparameter tuning was conducted using 5-fold cross-validation within the training dataset. For elastic net, the optimal alpha and lambda values were determined using a grid search. For the SVM, a radial basis function (RBF) kernel was applied, with both the cost (C) and the gamma parameters tuned. The number of trees and maximum depth were optimized for the random forest models. For models sensitive to class imbalance (e.g., logistic regression and SVM), class weights were adjusted accordingly [21,22,23,24,25].

The prediction performance of each model was evaluated using five standard metrics: accuracy, precision, recall, F1 score (the harmonic mean of precision and recall), and the area under the receiver operating characteristic curve (AUC) [26,27]. These metrics provide a comprehensive view of classification effectiveness, particularly in the context of imbalanced outcomes. To enhance model interpretability and identify key prognostic predictors of MACEs, we conducted a SHapley Additive exPlanations (SHAP) analysis on the best-performing model, namely, the random forest model [28,29]. SHAP values quantify the marginal contribution of each variable to the model’s predictions.

All the statistical analyses and modeling procedures were conducted using R (version 4.3.0) and Python (version 3.6.5). Key libraries included scikit-learn, xgboost, and SHAP in Python and caret, glmnet, and randomForest in R [14,30,31,32,33]. Random seeds were fixed to ensure reproducibility across iterations.

3. Results

3.1. Baseline Characteristics and Clinical Outcomes

From the COREA-AMI registry (n = 10,719 patients), a total of 6747 patients were included in the 5-year analysis, among whom 2061 (30.5%) experienced MACEs. For the 1-year analysis, 9624 patients were included, and 1087 patients (11.3%) experienced MACEs (Figure 1).

As shown in Table 1, patients who experienced MACEs at both 1 and 5 years were older and had a higher incidence of comorbidities such as hypertension, diabetes mellitus, atrial fibrillation, chronic renal disease, chronic obstructive pulmonary disease and malignancy. Procedural complexity, including left main PCI and long stenting, was also more frequent in the MACE group, whereas the use of intravascular ultrasound (IVUS) was less common. In contrast, lesion characteristics such as bifurcation, ostial, or chronic total occlusion lesions did not significantly differ between the groups. These trends were consistent across both time points.

3.2. Comparative Performance of Machine Learning Models

We compared the performance of four ML models—RF, SVM, elastic net, and NB—with that of traditional logistic regression for predicting MACEs at 1 and 5 years (Table 2 and Figure 2). The RF model consistently achieved the highest predictive performance across all the evaluation metrics, including the AUC, accuracy, and F1 score. At 5 years, the RF model achieved an AUC of 0.822, an accuracy of 0.804, and an F1 score of 0.870, which were superior to those of logistic regression (AUC 0.817, accuracy 0.798, F1 score 0.864). At 1 year, the performance gap became more prominent, with RF showing an AUC of 0.847, an accuracy of 0.930, and an F1 score of 0.962, compared with logistic regression (AUC of 0.843, accuracy of 0.927, and F1 score of 0.960). This superiority of the RF model was consistently observed in both the 1-year and the 5-year predictions.

Among alternative algorithms, elastic net and SVM showed comparable results to logistic regression, whereas the NB model demonstrated the lowest predictive performance, particularly for 5-year MACE (AUC 0.772; accuracy 0.755; F1 score 0.834).

3.3. Prognostic Predictors and SHAP Analysis

We further clarified the most influential predictors using SHAP analysis based on the RF model. The top 20 predictors for MACEs at 1 and 5 years according to the mean absolute SHAP value are shown in Figure 3. At 5 years, the top 5 predictors were age, estimated glomerular filtration rate (eGFR), dual antiplatelet therapy (DAPT), left ventricular ejection fraction (LV EF), and hemoglobin at 5 years. Similarly, at 1 year, DAPT was the most important predictor, followed by age, eGFR, statin use and beta-blocker use. The SHAP value plot revealed that older age, impaired renal function, the absence of DAPT, reduced LV EF, and anemia were associated with increased long-term adverse events. Notably, DAPT and statin use ranked higher at 1 year, highlighting the importance of optimal medical therapy for early prognosis.

4. Discussion

The major findings from the present analysis are as follows: (1) the incidence rates of 1-year and 5-year MACEs in our study are comparable to those reported in previous studies; (2) the RF model demonstrated slightly better prognostic performance compared with traditional statistical methods (logistic regression analysis); and (3) SHAP analysis enabled us to identify the independent predictors for both 1-year and 5-year MACEs.

At 5 years, MACEs occurred in 2061 patients (30.5%). At 1 year, MACEs occurred in 1087 patients (11.3%). These findings are consistent with those of prior studies. For instance, Kim Y et al. reported a 1-year adverse event rate (including all-cause mortality, MI, and any revascularization) of 12.3% in a national Korea AMI registry [34]. Similarly, a nationwide cohort study of 35,972 AMI patients who underwent PCI reported 4-year event rates of all-cause death and coronary revascularization ranging from 26.4% to 28.5% [4].

Although several studies have investigated ML-based prognostic models in AMI patients, many have been limited by their focus on all-cause mortality, in-hospital outcomes, or short-term follow-up [9,35,36]. In contrast, a key strength of our study is its focus on a specific composite of MACEs: cardiac death, MI, and cerebrovascular accidents. This approach offers a more direct assessment of long-term cardiovascular prognosis, as the endpoint is not influenced by noncardiac causes of death or by discretionary decisions such as revascularization. Furthermore, by evaluating risk for both 1-year and 5-year clinical outcomes, our study provides not only a more comprehensive perspective on long-term prognosis but also an understanding of the temporal dynamics of predictors. This large, multicenter, real-world nature of our PCI cohort enhances the generalizability of these findings, underscoring their relevance in routine clinical practice.

The RF model exhibited the highest predictive performance for both 1-year and 5-year MACE prediction in our analysis. Although many ML studies in clinical research report that model performance is primarily based on the AUC, this metric can be misleading in datasets with imbalanced outcomes. This is a common challenge in predicting clinical events such as MACEs, where the number of nonevents significantly outweighs the number of events. Recent studies have increasingly recognized this limitation, highlighting the need for more robust evaluation metrics [37,38].

Consistent with other real-world AMI registries, our study population exhibited significantly imbalanced outcomes, with a MACE incidence of 11.3% at 1 year and 30.5% at 5 years. In evaluating model performance under such imbalance, precision and recall are particularly important metrics, while AUC and accuracy also provide complementary information [26]. The precision (also called the positive predictive value) is the fraction of relevant instances among the retrieved instances, whereas the recall (also known as the sensitivity) is the fraction of relevant instances that were retrieved [39]. Therefore, the F1 score, which is derived from the harmonic mean of precision and recall, is particularly meaningful in our study [40]. We demonstrated that the F1 score of the RF model was the highest among all the ML techniques and logistic regression analyses.

Our study also provides valuable insights through SHAP analysis. A prior study by Oliveira M et al. employed SHAP to predict in-hospital mortality and identified variables such as cardiac dysrhythmia, glucose, cardiogenic shock, and renal failure [36]. However, many of these variables likely reflect effects of clinical deterioration rather than independent risk factors, thus limiting their use in long-term prognostic models. In contrast, our study excluded in-hospital complication variables such as cardiogenic shock, inotrope use, dialysis, transfusion, and acute heart failure, recognizing that these factors are not independent predictors but rather clinical consequences of deterioration status. As our goal was to identify modifiable predictors of long-term outcomes, we focused on baseline or discharge-level predictors that can inform long-term management strategies, particularly those related to secondary prevention.

Recently, Yang Y et al. established ML models for short-term (1 year) and long-term (5 years or 10 years) mortality prediction for AMI using SHAP analysis [37]. Using 4173 patients, the dataset was allocated into three groups (1-year group: 3626; 5-year group: 2102; and 10-year group: 721). Although our study shares some common findings with that study, a notable divergence emerges when we compare the top predictors identified by our SHAP analysis, such as ‘operation time’ or ‘apolipoprotein A’. These differences likely reflect the distinct endpoints and patient populations of the two studies. This study focused on all-cause mortality in a broader AMI population.

In contrast, our study was designed to predict a composite of MACEs within a well-defined cohort of post-PCI patients. Notably, the high ranking of predictors such as DAPT, statin, and LV EF in our analysis strongly aligns with current evidence-based clinical guidelines. This alignment underscores the clinical relevance and practical application of our model’s findings for optimizing patient care in a routine post-PCI setting.

Using this approach, we identified clinically relevant predictors of long-term outcomes in AMI patients. Among the modifiable factors, optimal medical therapy, such as dual antiplatelet therapy (DAPT), statins, and angiotensin-converting enzyme (ACE) inhibitors or angiotensin receptor blockers (ARBs), was consistently associated with a lower risk of MACEs. Beta-blockers also emerged as a predictor, although with relatively weaker importance. These findings are consistent with current clinical guidelines, which recommend extended DAPT and statin therapy for secondary prevention in AMI patients (class I recommendation) [41,42,43]. However, the long-term benefits of beta-blocker maintenance therapy remain a subject of ongoing debate, particularly in patients without left ventricular dysfunction or heart failure [44,45].

Interestingly, nonmodifiable predictors, such as age, renal function, hemoglobin level, and LV EF, ranked highest in terms of predictive power for both 1-year and 5-year MACEs. In contrast, the procedural variables (e.g., total stent length and mean stent diameter) had lower SHAP values, indicating a weaker prognostic influence. Although this may suggest that long-term prognosis is more strongly influenced by patient characteristics and medical therapy than by procedural details, it is important not to overinterpret these findings. Key procedural aspects, such as stent optimization or intravascular imaging use, which were not fully captured in our dataset, may play a significant role. Future studies incorporating these variables could better delineate the impact of procedural quality on long-term outcomes.

Our analysis has the following potential limitations: (1) This study was an observational registry study with inherent selection bias. (2) To accurately assess the prognostic performance of ML techniques for cardiovascular events, we excluded patients who died from noncardiac causes, which could limit the generalizability of our findings. (3) As the study population included only patients from South Korea, it is uncertain whether our findings can be directly applied to other ethnic groups. (4) The registry enrolled patients over a long period (2004–2015), during which time significant evolutions in stent technology and medical therapies occurred. These temporal changes in clinical practice could have acted as unmeasured confounders. (5) To reduce the risk of data leakage, variables directly related to the outcome (e.g., in-hospital adverse events) were excluded from both the imputation and model-building processes. Nevertheless, because imputation was conducted on the entire dataset before splitting into training and testing subsets, some leakage during model evaluation cannot be excluded, and unrecognized leakage may remain possible given the inherent complexity of registry datasets. (6) The absence of external validation is another limitation, as the predictive performance of our models was assessed only within the current dataset and may not be fully generalizable to other populations.

5. Future Research Directions

Future studies should aim to integrate procedural details—including the type of stent or drug-coated balloon—along with intravascular imaging (such as minimal stent area, stent optimization, and underexpansion), as well as longitudinal clinical data, to refine risk prediction and capture dynamic changes over time. Expanding outcome definitions beyond MACE to include endpoints such as heart failure hospitalization and quality of life will provide a more comprehensive, patient-centered perspective. In addition, our study demonstrated that a collaborative approach between statisticians and cardiologists can be instrumental in identifying and interpreting modifiable risk factors. By leveraging the RF model and SHAP analysis, these explainable AI methods can be translated into practical decision-support tools integrated with electronic records, thereby facilitating clinical implementation and optimizing individualized care.

Another important direction is benchmarking ML models against established clinical risk scores. The CADILLAC risk score is particularly relevant because it incorporates key procedural and clinical factors—including age, Killip class, anemia, renal function, multivessel disease, LVEF, and final TIMI flow—to predict long-term mortality after primary PCI. Integrating CADILLAC into future analyses could clarify whether ML provides incremental prognostic value beyond this validated risk score and help refine clinical decision-making for long-term outcomes.

A further promising avenue is the integration of cardiopulmonary exercise testing (CPET) with AI-driven analytics. CPET provides well-established prognostic markers such as peak oxygen consumption (VO_2_ max), ventilatory efficiency (VE/VCO_2_ slope), and oxygen pulse, while continuous ECG monitoring during exercise can reveal dynamic changes including ST-segment shifts, arrhythmias, and conduction abnormalities. Recent advances highlight the importance of explainable AI in cardiology, particularly in applications such as ECG interpretation and arrhythmia detection [46]. By applying ML and explainable AI to these multidimensional datasets, future studies may enhance long-term prognosis prediction, identify high-risk patient subgroups, and tailor individualized rehabilitation and follow-up strategies. Such an approach could bridge interventional cardiology with cardiac rehabilitation and preventive care, thereby advancing truly patient-centered cardiovascular management.

6. Conclusions

In this large, real-world cohort of AMI patients, the RF model demonstrated better prognostic performance over other ML techniques, while traditional logistic regression remained competitive. Furthermore, our analysis of predictive variables revealed that among modifiable risk factors, adherence to guideline-directed optimal medical therapy, critically including DAPT, statins, and ACE inhibitors/ARBs, was a key predictor of improved long-term outcomes at both 1 year and 5 years. These findings suggest that ML approaches may provide incremental value in identifying high-risk patients, while reaffirming the significant real-world impact of optimal medical therapy on the prognosis of patients after AMI.