Sina Score as a New Machine Learning-Derived Online Prediction Model of Mortality for Cirrhotic Patients Awaiting Liver Transplantation: A Prospective Cohort Study

TL;DR

This study introduces the Sina score, a new machine learning model that predicts three-month mortality in cirrhosis patients waiting for liver transplants.

Contribution

The Sina score is a novel machine learning-based mortality prediction model for cirrhotic patients.

Findings

The Sina score accurately predicted three-month mortality with an AUC of 0.753.

The Sina score outperformed the MELD score in predicting mortality for cirrhotic patients.

The model used anthropometric and clinical variables like hand grip and bilirubin levels.

Abstract

Background: Cirrhosis is responsible for a large proportion of mortality worldwide. Despite having multiple scoring systems, organ allocation for end-stage liver disease remains a major problem. Since anthropometric indices play important roles in predicting the prognosis of patients with cirrhosis, these variables were used in establishment of a novel scoring system. Methods: In order to evaluate a machine learning approach for predicting the probability of three-month mortality in cirrhotic patients awaiting liver transplantation, the clinical and anthropometric information of 64 patients referred to Abu-Ali-Sina Transplantation Center were collected and followed for three months. A LASSO logistic regression model was used to devise and validate a new machine learning approach and compare it to the Model for End-Stage Liver Disease (MELD) regarding the three-month mortality of cirrhotic patients. Hand grip, skeletal muscle mass index, average mean arterial pressure, serum sodium, and total bilirubin were assessed with this new machine learning approach to predict the prognosis of patients with cirrhosis, which we named the Sina score. Results: Sixty-four patients were enrolled, with a mean age of 46.50 ± 12.871 years. Like the MELD score, the Sina score is a precise prognostic tool for predicting the three-month mortality probability in cirrhotic patients [area under the curve (AUC) = 0.753 and p = 0.005 vs. AUC = 0.607 and p = 0.238]. Our machine learning approach, the Sina score, was shown to be a precise prognostic tool, like the MELD, for the prediction of the three-month mortality probability of cirrhotic patients awaiting liver transplantation. Conclusions: The Sina score, given that its level of precision is on par with the MELD, can be recommended for the prediction of three-month mortality in cirrhotic patients awaiting liver transplantation.

Figures

Full text

1. Introduction

Cirrhosis, or end-stage liver disease, affected more than 120 million people worldwide in 2017, while approximately 10 million suffered from decompensated cirrhosis [1]. The prevalence and incidence of cirrhosis is trending upwards due to lifestyle changes [2]. Cirrhosis led to about 1.5 million deaths in 2019 and accounts for 1.8% of the global incidence of disabilities per year [3,4]. Decompensated cirrhosis can also lead to other complications such as ascites, spontaneous bacterial peritonitis, esophageal variseal bleeding, hepatic encephalopathy, hepatorenal syndrome, and hepatocellular carcinoma [5].

Although the primary treatment for liver cirrhosis is managing the etiology and the complications of this condition, liver transplantation is still considered to be the best choice for select patients [6]. Indications for liver transplantation include cirrhosis in conjunction with decompensation. A Model for End-Stage Liver Disease (MELD) score of 15 or higher, or the presence of hepatopulmonary syndrome, are other indicators [7]. To address financial constraints and improve organ allocation, various models have been developed to assess the mortality risk of cirrhotic patients, such as the MELD [8]. The MELD is one of the most commonly used models for the prediction of the mortality of patients with cirrhosis, and new updates to it are being continually introduced [9].

The MELD scoring system utilizes serum creatinine and bilirubin and the International Normalized Ratio (INR) to assess the mortality risk of cirrhotic patients. However, recent studies have questioned its utility in the prediction of mortality in cirrhotic patients [10]. Further studies have shown that malnutrition is more severe in patients who expired within a shorter period of time [11]. Anthropometric parameters such as hand grip (HG), Mid-Arm Muscle Circumference (MAMC), and sarcopenia have been reported as indicators of malnutrition in cirrhotic patients [12].

Andrade et al. reported that diminished HG strength was related to a higher MELD score, demonstrating a correlation with a worse clinical outcome. The presence of decreased muscle strength in cirrhotic patients was illustrated to be linked to prognostic factors that can be valued as clinical data in the management of cirrhotic patients [13]. Ramachandran et al. evaluated the prevalence of malnutrition measuring HG and MAMC among cirrhotic patients referred for liver transplantation. They showed moderate-to-severe malnourishment, confirmed by low MAMC and HG measurements, which was associated with a MELD score of >15 and mortality [14]. Sarcopenia was shown to be associated with mortality in patients with cirrhosis. It is not correlated with the degree of liver dysfunction assessed by utilizing conventional scoring systems. Scoring systems including the assessment of sarcopenia are needed to better predict mortality in patients with cirrhosis [15]. In light of the previously mentioned evidence, and as anthropometric indices play important roles in predicting the prognosis of patients with cirrhosis [16,17], this study evaluated a new machine learning approach to predicting the three-month mortality probability in cirrhotic patients awaiting liver transplantation.

2. Materials and Methods

2.1. Devising a Prediction Scale in Cirrhosis

This prospective cohort study used a machine learning method that employed regularized Least Absolute Shrinkage and Selection Operator (LASSO) logistic regression to devise a prediction scale for three-month mortality in cirrhotic patients. The inclusion criteria for enrolled patients were as follows: being aged 18 years or older, having a Child–Turcotte–Pugh (CTP) score of 7 or higher, and having been referred to the Abu-Ali Sina Transplantation Center in Shiraz, Iran. The patient exclusion criteria were as follows: those who were younger than 18 years, those afflicted with HIV/AIDS, those suffering from Chronic Kidney Disease (CKD) and in need of Renal Replacement Therapy (RRT) such as hemodialysis, those afflicted with chronic pulmonary disease and in need of supplementary oxygen, pregnant and breastfeeding individuals, patients finding a liver donor and undergoing liver transplantation, and, finally, patients unwilling to participate in the study. All data were collected from the files of patients referred to Abu-Ali Sina Transplantation Center, Shiraz, Iran.

Since no previous study had employed our model in a similar context, the calculation of sample size was made based on a comparison of the Skeletal Muscle Mass Index (SMI) between surviving and deceased cirrhotic patients. Considering a type I error of 0.05 and a study power of 80% for male patients, the minimum sample size was estimated to be six participants. In this study, a census method was utilized and patients who needed a liver transplant during the one-year period were evaluated, which finally resulted in an adequate pool of patients. This study was approved by the Deputy of Research and Technology and the Committee of Ethics in Biomedical Research of Shiraz University of Medical Sciences, with the code IR.SUMS.REC.1400.220. After explaining the study to the patients and obtaining written consent, the eligible patients were enrolled in the study.

During physical examination, Mid-Arm Muscle Circumference (MAMC) was measured, and hand grip (HG) strength was calculated using a dynamometer. Average mean arterial pressure (AMAP) was determined using 24 h Holter monitoring. Moreover, the patients underwent spiral abdominopelvic CT scans, in which their SMI score was assessed. Since abdominopelvic CT scanning is routinely performed for cirrhotic patients, this procedure did not impose additional cost or radiation on the patients. Laboratory data such as serum creatinine, blood urea nitrogen (BUN), complete blood count (CBC), erythrocyte sedimentation rate (ESR), International Normalized Ratio (INR), serum sodium and potassium, and the complete liver function test (LFT) were evaluated. Then the patients were medically followed for three months.

2.2. The Inclusion and Exclusion Criteria

Patients aged 18 years or older with a Child–Turcotte–Pugh (CTP) score of 7 or higher who were referred to Abu-Ali Sina Transplantation Center were included in the study. The exclusion criteria were as follows: age under 18 years, affliction with HIV/AIDS, chronic kidney disease (CKD) requiring renal replacement therapy (RRT) such as hemodialysis, chronic pulmonary disease in need of supplementary oxygen, pregnancy, breastfeeding, and unwillingness to participate in the study. Patient recruitment occurred from December 2020 to September 2021, with medical follow-ups lasting until December 2021. The recruitment of the validation group of 21 patients lasted from January 2022 to April 2022, with follow-up extending to July 2022.

2.3. Statistical Analysis

The data were analyzed using the R statistical program (version 3.6.3, R Foundation for Statistical Computing). The quantitative variables were reported as means and standard deviations, while the qualitative data were reported as frequencies and percentages. Comparisons of quantitative variables between the deceased and surviving patient groups were conducted using independent two-sample t-tests, and categorical variables were compared using the chi-square test.

Variable selection was performed using regularized LASSO logistic regression, a machine learning method. Variables with non-zero coefficients (β) were included in the model, while variables with zero coefficients (β) were excluded (Table 1). Then, the maximum absolute beta value for each variable was calculated using the formula of $[eqn]$ (Table 2).

The maximum absolute beta values were ranked first to kth (where k is the number of selected variables using the LASSO logistic regression model). The maximum absolute beta value of the top-ranked variable was used as the denominator for calculating each variable’s point contribution. The risk score equation was constructed by a linear interpolation method (Table 2). All statistical analyses were performed using the “rms” and “penalized” packages in R statistical software. The “rms” package of the R statistical program developed a graphic illustration for each variable’s point contribution and the overall risk score (Figure 1). Receiver Operating Characteristic (ROC) curves and their Area Under the Curve (AUC) values were used to compare the performance of the newly developed risk score with the MELD score (Figure 2). p-values of less than 0.05 were considered statistically significant.

This novel machine learning-based scoring system was named the Sina Score, in honor of the Persian physician Avicenna (Ibn Sina) who lived in the 10th and 11th centuries. To validate this new scoring system, 21 additional patients referred to our center were evaluated by both the Sina and MELD scores. ROC curves were generated for all participants to evaluate predictive accuracy.

3. Results

3.1. Demographic, Anthropometric, and Laboratory Findings

Among the 64 enrolled patients, 41 (64.1%) were male and 23 (35.9%) were female. The mean age of the patients was 46.50 ± 12.871 years. The patients were followed up for a period of three months, during which 13 (20.3%) died and 51 (79.7%) survived. The demographic, anthropometric, and laboratory findings for both groups are summarized in Table 1. As can be seen in Table 1, deceased patients were significantly older than survivors (53.77 ± 9.293 vs. 44.25 ± 13.252 years old, p = 0.006), had weaker hand grips (16.51 ± 8.682 vs. 24.14 ± 10.517 kg, p = 0.013) and had lower serum albumin and sodium levels (p = 0.049 and 0.022, respectively). The demographic, clinical, and laboratory results were assessed by univariate binary logistic regression analysis to predict mortality in cirrhotic patients. As can be seen in Table 1, older age (odds ratio [OR] = 1.073, p = 0.025), lower hand grip strength (OR = 0.913, p = 0.025), and lower sodium levels (OR = 0.874, p = 0.030) were significantly associated with increased risk of mortality. The results of the univariate analysis are summarized in Table 1.

3.2. The Prediction Scale in Cirrhosis

Since the sample size was low, regularized LASSO logistic regression was used to assess the key predictors of mortality in cirrhotic patients. Five variables turned out to have non-zero coefficients in LASSO regression analysis that entered our model, including HG, SMI, AMAP, Na, and total bilirubin (β column of Table 2). The risk of death was calculated using the formulae of linear predictors as follows:

Linear predictor = 19.8846 − 0.0956 × HG (kg) + 0.0351 × SMI − 0.0308 × AMAP (mmHg) − 0.1324 × Na (mEq/lit) − 0.0520 × Total bilirubin(mg/dL)

$$[eqn]$$

The results of this regression model were used to devise a new risk score by utilizing the maximal absolute coefficients of the variables. Table 2 summarizes the calculation of the points assigned to each variable in the risk score. Based on the findings in Table 2, the novel risk score was calculated using the following formula:

Risk Score = 624.69 − 2.0408 × HG + 0.7493 × SMI − 0.6575 × AMAP − 2.8263 × Sodium − 0.2772 × Total bilirubin.

3.3. Sina Scoring System

To increase the usability of the risk score, a nomogram for the Sina scoring system was generated by the R program (Figure 1) that helps estimate the risk of mortality on a visual scale. To compare the Sina score risk assessment tool with the MELD, the ROC and AUC were used and presented in Figure 2. The AUCs for Sina and MELD scores were 0.753 (95% Confidence Interval [95% CI]: 0.630–0.851) and 0.607 (95% CI: 0.478–0.725), respectively, showing the superior performance of the Sina score in predicting the risk of mortality in cirrhotic patients. At an optimal cutoff point of 195.6835, the Sina score could predict mortality with a sensitivity of 53.8% (95% CI: 25.2–80.7) and a specificity of 94.2% (95% CI: 84.0–98.7). The model had a 70.0% positive predictive value and an 89.1% negative predictive value. The high specificity of the Sina score has made it a potent risk stratifying tool for organ allocation in cirrhotic patients needing liver transplantation.

A linear interpolation method was used in the R statistical program to validate the Sina scoring system and draw a survival function graph. The graph drawn by the program showed a clear inverse relationship between the Sina score and three-month survival of cirrhotic patients. The linear interpolation graph is presented in Figure 3.

To assess the validity of the risk scoring system in the context of cirrhotic patients, a validation cohort of 21 cirrhotic patients was subsequently analyzed. The mean age of the validation group was 50.57 ± 12.221 years, and a male-to-female ratio of 5:2 was seen (71.4% of the patients were male, while 28.6% were female). No significant difference was seen between the development group and the validation group regarding age (p = 0.159) and sex (p = 0.485). Among the patients in the validation group, only two expired during a three-month interval. An ROC curve for the prediction of mortality using Sina and MELD scores was drawn for this population. The AUCs for MELD and Sina scores in the validation group were 0.706 (p = 0.173) and 0.825 (p = 0.032), respectively, demonstrating the superior predictive accuracy of the Sina scoring system. Figure 4 shows the ROC curve of the validation group.

4. Discussion

In this study, we evaluated a machine learning approach to predict mortality in patients with end-stage liver disease, aiming to improve organ allocation decisions in liver transplantation centers. The proposed Sina scoring system incorporates HG, SMI, AMAP during Holter monitoring, serum sodium, and total bilirubin levels.

Sarcopenia, defined as the loss of skeletal muscle mass, has been shown to be associated with chronic liver disease and an increase in the mortality of cirrhotic patients [18,19]. While esophageal variseal bleeding, hepatorenal syndrome, and hepatic encephalopathy are important predictors of acute mortality, sarcopenia reflects a more chronic deterioration in physiological reserve. Anthropometric indices such as body mass index (BMI), SMI, HG, and MAMC are utilized to assess sarcopenia and the nutritional status of the patient; thus, theses parameters were used in the current study to predict mortality in cirrhotic patients [20,21,22].

Furthermore, lower MAP was found to be independently associated with higher mortality in cirrhotic patients [23]. Consequently, we used the above-mentioned features to evaluate cirrhotic patients in our study. Serum sodium, total bilirubin and creatinine levels, and INR were also measured to evaluate the patients’ MELD scores as the current gold standard of prognostic evaluation in cirrhotic patients. Mallik et al. found that although the MELD is universally used for decision making in organ allocation for liver transplantation, the Chronic Liver Failure–Sequential Organ Failure Assessment (CLIF-SOFA) scoring system is superior in the prediction of short-term mortality among liver cirrhosis patients [24].

Our findings reveal that the MELD scoring system alone may impair medical judgment to determine the risk of mortality in cirrhotic patients. Although the MELD is globally used for the prediction of mortality in patients with cirrhosis, the addition of other variables in this scoring system can enhance its accuracy, such as adding C-Reactive Protein (CRP) and procalcitonin, which was demonstrated to improve the precision of mortality prediction in cirrhotic patients [25]. Similarly, Rodríguez-Perálvarez et al. found that the Gender-Equity Model for liver allocation corrected by serum sodium (GEMA-Na) and the Model for End-Stage Liver Disease 3.0 (MELD 3.0) could correct sex disparities for accessing liver transplantation and provide more accurate predictions of waiting list outcomes [26]. Also, the addition of lactate to the variables measured with the MELD score may make it superior in predicting mortality [27]. Deng et al. found that incorporating a new variable, such as five-meter gait speed as a measurement of frailty, in the MELD score could be associated with an increased predictive accuracy for mortality among cirrhotic patients [28]. The MELD scoring system incorporates serum creatinine levels to predict cirrhosis mortality, which is directly correlated with the muscle mass index of the patients; however, in our study, the SMI and HG were included for nutritional assessments of the patients, which could be the primary changes before creatinine alterations [29]. Moreover, Montano-Losza et al. performed a study on 669 cirrhotic patients and found out that the embodiment of sarcopenia assessed by the SMI at the level of third lumbar vertebra can increase the prognostic power of the MELD score for mortality [30].

In our study, malnutrition and frailty were measured by anthropometric indices, which were integrated into the Sina score. In a study performed by Tapper et al. on 274 patients with a 5-year follow-up, they found that body composition including, muscle and fat distribution, could outperform the MELD in predicting the risk of mortality and decompensation in cirrhotic patients. They found that fat density was the most important factor to predict decompensation and mortality in patients with end-stage liver disease [31]. These findings confirm our results regarding the importance of anthropometric indices in the assessment and prognosis of mortality in cirrhotic patients.

In another study performed by Kanwal et al., the authors developed new models for the prediction of mortality in cirrhotic patients by using LASSO regression containing numerous variables, such as hospitalization due to cirrhosis and the number of outpatient visits [32]. Our Sina score model, unlike the model used by Kanwal et al., consists of five core clinical parameters and provides a clear, usable formula for risk calculation, which increases its potential for bedside application. In a study by Maharshi et al., the authors found that the prevalence of malnutrition increased with higher CTP scores. Furthermore, the complications and mortality of cirrhosis in malnourished patients were more than that of well-nourished patients, revealing that nutritional assessments of cirrhotic patients can be of prognostic value in determining the risk of complications and mortality [33]. These findings denote the baseline concept that was used in the development of the Sina score as a new model for mortality risk assessment in cirrhotic patients.

Guo and colleagues conducted a study on a database of 34,575 cirrhotic patients utilizing multiple machine learning and deep learning methods to devise new prognostic models for mortality. These new models outperformed the conventional MELD scoring system in 3-month, 6-month, and one-year periods. However, since they did not provide a clear formula for the assessment of mortality risk, the clinical significance of the study is questionable, and using such data seems infeasible [34]. In contrast, our model retains interpretability crucial for clinical decision making. Finally, Chang et al. performed a meta-analysis on muscle mass loss and mortality in cirrhotic patients with a pooled population size of 4070 participants. They concluded that the loss of muscle mass had an adjusted hazard ratio of 2.36 in cirrhotic patients and an odds ratio of above 1.5 for advances in CTP class. Moreover, it was shown that patients with loss of muscle mass had a three-fold increase in infection risk [35]. As a result, the SMI as a measurement of muscle mass was demonstrated to be utilized as a means of mortality risk assessment.

5. Conclusions

In conclusion, we introduced and validated a new machine learning-based prognostic tool named the Sina score to predict three-month mortality in cirrhotic patients and help optimize transplant resources in decision making for liver allocation. By incorporating clinically relevant variables—including hand grip strength, skeletal muscle index, mean arterial pressure, serum sodium, and total bilirubin—this model demonstrated superior predictive accuracy compared to the traditional MELD score. These findings can improve risk stratification and patient selection for orthotopic liver transplantation.