Survival Meets Classification: A Novel Framework for Early Risk Prediction Models of Chronic Diseases

TL;DR

This paper introduces a novel framework that combines survival analysis with classification to improve early risk prediction of chronic diseases using big EMR data, achieving comparable or superior performance to existing models.

Contribution

It presents a new integrated approach that re-engineers survival analysis methods for classification, providing a comprehensive tool for disease risk prediction and explanation generation.

Findings

Survival models perform as well as or better than LightGBM and XGBoost.

The models provide clinically validated explanations.

The approach effectively handles big EMR datasets.

Abstract

Chronic diseases are long-lasting conditions that require lifelong medical attention. Using big EMR data, we have developed early disease risk prediction models for five common chronic diseases: diabetes, hypertension, CKD, COPD, and chronic ischemic heart disease. In this study, we present a novel approach for disease risk models by integrating survival analysis with classification techniques. Traditional models for predicting the risk of chronic diseases predominantly focus on either survival analysis or classification independently. In this paper, we show survival analysis methods can be re-engineered to enable them to do classification efficiently and effectively, thereby making them a comprehensive tool for developing disease risk surveillance models. The results of our experiments on real-world big EMR data show that the performance of survival models in terms of accuracy, F1 score, and AUROC is comparable to or better than that of prior state-of-the-art models like LightGBM and XGBoost. Lastly, the proposed survival models use a novel methodology to generate explanations, which have been clinically validated by a panel of three expert physicians.