Evaluation of the Cleveland Clinic Score for predicting acute kidney injury across different elective cardiac surgeries—a retrospective study

TL;DR

This study evaluates how well the Cleveland Clinic Score predicts kidney injury after different heart surgeries, finding moderate accuracy overall but suggesting improvements for specific procedures.

Contribution

The study validates the Cleveland Clinic Score's predictive ability for AKI across various cardiac surgeries and highlights the need for modifications in specific cases.

Findings

The Cleveland Clinic Score showed moderate predictive ability for AKI (AUC 0.630) across all cardiac surgeries.

The score was most accurate for coronary artery bypass grafting and aortic valve procedures.

The score was particularly useful in predicting the need for renal replacement therapy.

Abstract

Acute kidney injury (AKI) after cardiac surgery is a serious postoperative complication associated with an increased risk of mortality. The Cleveland Clinic Score (CCS) is one of the tools that allows preoperative assessment of the likelihood of developing AKI. However, the tool has not been validated in different types of cardiac surgery procedures. Our aim was to evaluate the CCS before different types of cardiac surgery and to assess the usefulness of this tool as a predictor of AKI.

In this retrospective study we included patients who underwent elective cardiac surgery in 2023. Our endpoint was AKI, as defined by the Kidney Diseases Improving Global Outcomes (KDIGO) criteria. The predictive value for AKI after cardiac surgery (CCS) was evaluated using receiver operating characteristic (ROC) curves and area under the curve (AUC) values.

A total of 610 patients underwent elective cardiac surgery. Patients with and without AKI were divided into CCS stages: stage I (57.8 vs 72.3%), stage II (36.1 vs 26.1%), stage III (5.4 vs 1.6%), stage IV (0.6 vs 0%). The AUC for all operations was 0.630 (95% CI [0.580–0.679], p < 0.001), stage I 0.428 (95% CI [0.376–0.480]; p = 0.006), stage II 0.550 (95% CI [0.498–0.602]; p = 0.057) and for stage III 0.519 (95% CI [0.467–0.572]; p = 0.464). The AUC values were significant only for coronary artery bypass grafting (CABG) 0.650 (95% CI [0.552–0.748]) and aortic valve replacement/plasty (AVR/AVP) 0.629 (95% CI [0.550–0.709]).

The overall CCS value showed a moderate predictive ability for AKI (AUC 0.630) and particularly useful for predicting renal replacement therapy (RRT), but for individual groups the scale should be modified by adding several new factors.

Figures

ROC curve.

ROC curve with bootstrap analysis.

ROC curves for different surgery types.

Full text

Introduction

Acute kidney injury (AKI) is a serious complication that can occur after cardiac surgery. Depending on the author and definition used, 5–30% of cardiac surgery patients may develop AKI. Exposure to nephrotoxic drugs and substances, such as aminoglycosides and radiocontrast agents, may increase the risk of developing AKI (Chertow et al., 1997; Rosner & Okusa, 2006; Huen & Parikh, 2012; Bhat et al., 1976; Gailunas et al., 1980; Corwin et al., 1989). The reported pathophysiological mechanisms for AKI caused by cardiac surgery include ischemic injury, renal reperfusion syndrome, inflammation, atheroembolism, neurohormonal activation, and oxidative stress (Sethi et al., 1987; Perella et al., 1992; Wan, LeClere & Vincent, 1997; Doty et al., 2003). The risk of mortality associated with AKI after open heart surgery is 5 times higher in patients with AKI than in patients without AKI (Thakar et al., 2007; Rao et al., 2018; Korczak et al., 2022).

The use of less invasive cardiac surgical techniques and off-pump coronary artery bypass grafting has resulted in lower mortality rates and incidences of acute renal failure among patients undergoing minimally invasive and/or off-pump surgery. However, post-operative renal dysfunction has remained unchanged in conservative open-heart surgery. After cardiac surgery, one to five percent of patients require dialysis for AKI, and this condition is strongly associated with perioperative morbidity and mortality (Zhang et al., 2008; Swaminathan et al., 2007). The mortality rate of patients with AKI requiring dialysis is estimated to be greater than 50% (Lopez-Delgado et al., 2013). AKI associated with cardiac surgery increases the risk of infection and prolongs hospital and intensive care unit stays (Bove et al., 2004; Lassnigg et al., 2004). This increases the use of healthcare resources and is associated with higher mortality (Ryckwart et al., 2002).

Several risk models have been developed to estimate the risk of postoperative kidney injury after cardiac surgery. Among these models, the Cleveland Clinic model is the most widely tested, and according to several studies, it has the highest discriminative power in most populations tested  (Huen & Parikh, 2012; Englberger et al., 2010). The Cleveland Clinic Score (CCS), developed by Thakar et al. (2005), is a clinical tool to predict AKI risk after cardiac surgery, incorporating factors such as female gender, congestive heart failure, left ventricular ejection fraction (LVEF) <35%, preoperative intra-aortic balloon pump (IABP) use, chronic obstructive pulmonary disease (COPD), insulin-dependent diabetes, previous cardiac surgery, emergency surgery, type of surgery, and preoperative creatinine levels, with scores ranging from 0 to 17 points (Table 1). It can be used in the preoperative evaluation of AKI and is appropriate for all patients undergoing cardiac surgery. Predicting the development of AKI, its early treatment and prevention are goals of cardiac surgeons and nephrologists involved in the care of these patients, providing an opportunity to develop strategies for early diagnosis and treatment. Recent studies have proposed enhancing the CCS by incorporating additional predictors such as baseline hemoglobin, estimated glomerular filtration rate (eGFR), and glycosylated hemoglobin (HbA1c), which may improve its predictive accuracy across diverse patient populations (Vives et al., 2024).

Table 1: Components of the Cleveland Clinic score.

The purpose of the study was to evaluate the Cleveland Clinic Score before different types of surgery and to assess its usefulness as a predictor of acute kidney injury after elective cardiac surgery.

Materials & Methods

Study design and participants

In this retrospective study, we enrolled 610 patients aged 18 years and older who underwent cardiac surgery between January and December 2023. For this retrospective study, we enrolled 610 patients aged 18 years and older who underwent cardiac surgery between January and December of 2023. We excluded patients who were younger than 18 years old, those who underwent non-elective surgery, and those whose type of surgery was performed fewer than 10 times during this period. If a patient underwent more than one cardiac surgery during the same hospitalization, only the data from the first surgery were considered.

We reviewed the databases and medical records of enrolled patients, collecting the data on their demographic details and medical histories, including important clinical, operative and peri-operative data, such left ventricular ejection fraction (LVEF), pre- and post-operative serum creatinine level, estimated glomerular filtration rate (eGFR), European System for Cardiac Operative Risk Evaluation (EuroSCORE), type of surgery, postoperative complications and treatment, and CCS. The CCS was calculated retrospectively for each patient based on preoperative data extracted from medical records, using the standard formula outlined by Thakar et al. (2005) (Table 1).

We could not distinguish between type 1 and type 2 diabetes, so we grouped all diabetic patients into a single category. Because our study was limited to patients with elective surgery, we excluded emergency surgery as a parameter in the Syntax Clinic Score.

Our endpoint was AKI as defined by the Kidney Diseases Improving Global Outcomes (KDIGO) criteria. AKI stage 1 was identified by an increase in serum creatinine of ≥ 0.3 mg/dL (≥26.4 µmol/L) or an increase of 1.5–1.9 times baseline. AKI stage 2 was defined by an increase in serum creatinine of 2–2.9 times baseline. AKI stage 3 was defined by a 3-fold increase in baseline serum creatinine, a serum creatinine increase to four mg/dL (353.6 µmol/L), or the need for renal replacement therapy (RRT). Due to data limitations and the potential effect of postoperative diuretic use, urine output criteria were not included in the KDIGO definition of AKI. This may have led to underdiagnosis of AKI, particularly in cases where urine output was reduced without significant creatinine changes, as urine output is a sensitive early indicator of AKI.

Ethics

This study was approved by the Research Ethics Committee of the Medical College of Jagiellonian University (approval number: 118.0043.1.119.2024) and was conducted in accordance with the Declaration of Helsinki. The written informed consent was obtained from the participants upon their admission to the hospital.

Statistical analysis

In this retrospective study, an inferential statistical analysis was performed using SPSS Statistics V29.0 (IBM Corp., Armonk, NY, USA). Categorical variables were compared between groups using Fisher’s exact test or the chi-2 test and presented as counts and percentages. Differences in continuous variables were tested with the Mann–Whitney U test, presented as median along with interquartile range (IQR) or with the t-Student test, presented as mean with standard deviation (SD).

The test characteristics (sensitivity, specificity, positive and negative predictive values) of CCS as a predictor of AKI were evaluated by constructing ROC curves and calculating the corresponding area under the curve (AUC) values. Statistical significance was determined at a p-value of less than 0.05.

Results

Between January and December of 2023, a total of 610 patients were admitted for elective surgery (see Table 2). Of those patients, AKI of any stage occurred in 166 patients (27.2%) after elective cardiac surgery. Patients without AKI had fewer comorbidities, such as arterial hypertension, diabetes mellitus, chronic kidney injury, atrial fibrillation, and carotid artery disease, compared with patients with AKI. They were also significantly younger (65 vs. 69 years, p < 0.001) and had a higher estimated glomerular filtration rate (eGFR). There were no significant differences between the AKI and non-AKI groups with respect to sex, chronic pulmonary disease, preoperative anemia, coronary or peripheral artery disease, history of stroke, smoking status, body mass index, left ventricular ejection fraction, or type of surgery. Patients with elevated baseline serum creatinine levels and a higher EuroSCORE II score were more likely to develop AKI after cardiac surgery (p < 0.001).

Table 2: Baseline characteristics for patients with and without postoperative AKI.

During the postoperative period, re-thoracotomy, the need for blood transfusions in the ICU, and the need for intravenous inotropic support were more frequent in the AKI group.

Patients with AKI after cardiac surgery, were also more likely to develop postoperative pulmonary complications (including respiratory failure, pleural fluid and pneumonia) and surgical site infection (including deep wound infection necessitating drainage and resuturing). They also spent more time in the hospital (both ICU and postoperative clinic) and had higher hospital mortality. Continuous veno-venous hemodialysis (CVVHD) was used in 15 patients (9%) with AKI and in 2 (0.5%) without AKI (p < 0.001). In patients without AKI, CVVHD has been used for the management of peripheral fluid retention due to postoperative left ventricular failure.

The Cleveland Clinic Scores were significantly higher in the AKI group. Using ROC analysis, CCS showed predictive ability for AKI with need for RRT in all patients with an AUC of 0.630 (95% CI [0.580–0.679]; p ≤ 0.001) (Figs. 1 and 2, Table 3). The results indicate that the Cleveland Clinic score has a high sensitivity of over 70%, with a relatively low specificity of over 40%. This means that it detects the disease with high sensitivity in most patients, but may produce false-positive results in healthy individuals. This is a valuable clinical observation, as it means that the diagnosis of an increased risk of AKI based on CCS requires additional confirmation with other tests and laboratory results. The values were statistically significant for CABG (AUC 0.650; 95% CI [0.552–0.748]; p = 0.003) and aortic valve replacement/plasty (AVR/AVP) (AUC 0.629; 95% CI [0.550–0.709]; p = 0.002). In the case of predictive analysis for CABG and AVR/AVP, for which statistical significance was demonstrated, sensitivity and specificity were 63%, 60% and 75%, 48%, respectively. CCS was predictive for AKI stages II (AUC 0.550; p = 0.057) and III (AUC 0.519; p = 0.464) (Fig. 3, Table 4). The better performance of CCS for CABG and AVR/AVP may be due to shared pathophysiological mechanisms, such as atherosclerosis in coronary artery disease and aortic stenosis, which are captured by CCS components like preoperative creatinine and LVEF (Table 1).

ROC curve.

ROC curve with bootstrap analysis.

Discussion

The true incidence of acute kidney injury (AKI) in patients undergoing cardiac surgery is unclear because different authors have used different terminology to define it. In a South Asian cohort of 276 patients, the overall incidence of AKI, as defined by the KDIGO criteria, was 6.88% (Rao et al., 2018). Wong et al. (2015) reported AKI in 14.5% of patients after cardiac surgery. Robert et al. (2010) analyzed a group of 25,086 patients undergoing cardiac surgery in northern New England. According to the Acute Kidney Injury Network (AKIN) criteria, AKI occurred in 30% of patients, and according to the Risk, Injury, Failure, Loss, and End-stage kidney disease (RIFLE) criteria, it occurred in 31%. In our study, acute kidney injury of any stage occurred in 27.2% of patients after elective cardiac surgery according to the KDIGO criteria.

In our study, we identified the following preoperative risk factors for AKI development: greater number of comorbidities, older age, and higher European System for Cardiac Operative Risk Evaluation (EuroSCORE) II score. EuroSCORE is a cardiac risk model that predicts 30-day mortality after cardiac surgery. Derived from an international European database, it was first introduced in 1999 and updated to its second version, EuroSCORE II, in 2011 (Nashef et al., 2012).

Preoperative renal impairment has been proven to be an independent risk factor for postoperative mortality. Therefore, creatinine clearance (CCr) was incorporated into EuroSCORE II with three levels of renal insufficiency: moderate (CCr 50–85 mL/min), severe (<50 mL/min), and on dialysis (irrespective of CCr). Thus, elevated EuroSCORE values in AKI patients may reflect reduced preoperative renal function. AKI patients were also more likely to undergo rethoracotomy, receive blood transfusions, and require inotropic support during the perioperative period. All of these postoperative factors are associated with transient renal ischemia. It has been shown that early postoperative AKI after cardiac surgery is strongly associated with two major factors: reduced functional reserve and renal ischemia.

AKI is a frequent complication following cardiac surgery, increasing the risk of mortality, morbidity, prolonged hospital stays, and hospital costs. Identifying patients at high risk for postoperative kidney damage following cardiac surgery is the first step in preventing this complication. Several scoring models have been developed to facilitate risk stratification and improve clinical decision-making (Pannu et al., 2016; Alhulaibi et al., 2022; Chen et al., 2016; Jiang et al., 2017; Vogt et al., 2021; Dedemoğlu & Tüysüz, 2020; Elmedany et al., 2017).

In our study, we used the CCS, a prognostic tool that assesses the risk of postoperative acute kidney injury. However, it should be emphasized that both the Cleveland Clinic score and the Euroscore depend heavily on preoperative serum creatinine levels, which are the most influential factor. The Canadian study by Wong et al. (2015) evaluated the CCS’s ability to predict both AKI requiring dialysis and less severe stages of AKI in 2,316 patients from a tertiary care center. The study found that the CCS was valid in identifying patients with severe stages of AKI but had less discriminative power for earlier stages.

Table 3: Area under the curve values of different types of surgeries.

Our novel approach was to apply this score to the general cardiac surgical population and evaluate its usefulness with respect to different types of surgery (e.g., CABG, valve surgery). Our analysis suggests that using the CCS to estimate risk by type of surgery may provide more accurate AKI predictions. Most studies have focused on the general use of the CCS score without considering the diversity of surgical procedures, and only a few have differentiated CCS scores for specific types of surgery, such as coronary artery bypass grafting (CABG), valve surgery, and other surgeries (Huen & Parikh, 2012).

In our analysis, for the entire study, the AUC ROC CCS was 0.63 (0.580−0.679; 95% CI).

Additionally, we confirmed these results in a bootstrap analysis. When analyzing the clinical usefulness of the test, we assessed that Cleveland Clinic Score has a high sensitivity of over 70%, with a relatively low specificity of over 40%. This means that it detects the disease with high sensitivity in most patients, but may produce false-positive results in healthy individuals.

Our approach revealed that different types of surgery are associated with distinct risk profiles that the CCS can effectively capture with the proper adjustments. ROC analysis demonstrated the statistical power to predict AKI in patients undergoing CABG (AUC 0.650; p = 0.003) and aortic valve replacement/repair (AVR/AVP) (AUC 0.629; p = 0.002) (Fig. 1, Table 3). The pathophysiological mechanisms of AKI after cardiac surgery include ischemia and atheroembolism. Atherosclerosis is the common pathological basis of coronary artery disease and aortic stenosis. This may explain the importance of the CCS in predicting AKI in patients undergoing CABG and aortic valve surgery. However, it should be noted that the small group size may limit the reliability of these findings.

ROC curves for different surgery types.

Table 4: Area under the curve values of different types of surgeries divided into Cleveland Clinic Score stages.

Many studies propose including additional factors in the scale, such as urine biomarkers (e.g., neutrophil gelatinase-associated lipocalin, interleukin-6, hepcidin-25, and midkine), age, body mass index (BMI), and the incidence of arrhythmia and pulmonary hypertension.; serum markers such as lactate, total bilirubin, albumin, white blood cells, cystatin C, and uric acid; cardiopulmonary bypass time; aortic cross-clamp duration; and central venous pressure (Albert et al., 2020; Yan et al., 2023; Huang et al., 2022; Wang et al., 2022; Che et al., 2019; Palomba et al., 2007; Krzanowska et al., 2024). Introducing these factors into clinical practice could lead to better risk management and improved patient outcomes during surgical procedures.

CCS is primarily used to evaluate the risk of postoperative RRT. In our study, we found that CCS grades 2 and 3 values are just above the reference line, indicating an increased risk of RRT. However, grade 1 is well below this line. These results align with the CCS scale’s assumptions, confirming that the total score and individual stages are effective prognostic tools, particularly for identifying patients requiring postoperative RRT.

It should be noted, however, that further studies are needed to fully verify the usefulness and accuracy of the CCS in different surgical contexts. Therefore, our results may serve as a foundation for future studies aimed at refining the CCS scale to meet the unique requirements of various surgical procedures. Our findings suggest that clinicians can use the CCS scale to identify patients at higher risk for acute kidney injury requiring renal replacement therapy, particularly for coronary artery bypass grafting and aortic valve replacement/aortic valve plasty, enabling targeted preoperative optimization. For milder AKI stages or other surgeries, however, alternative or more refined risk models may be necessary.

Study limitations

Our analysis has several limitations. First, it is a retrospective, single-center study. Second, we included a limited number of patients. Third, we only used creatinine criteria and did not include urine criteria due to data limitations and the postoperative diuretic effect.

Conclusions

Our study found that the Cleveland Clinic Score was a moderate predictor of acute kidney injury and was especially useful in identifying patients who needed renal replacement therapy, especially those undergoing CABG and AVR/AVP procedures. According to our findings, the predictive accuracy of the CCS may be enhanced for different stages of AKI and cardiac surgery types by taking into account variables like advanced age, preoperative estimated glomerular filtration rate, and comorbidities like diabetes and hypertension, which demonstrated strong correlations with AKI.

Supplemental Information

10.7717/peerj.20533/supp-1Supplemental Information 1Raw data

10.7717/peerj.20533/supp-2Supplemental Information 2Codebook for raw data