From raw data to actionable insights: preprocessing real-world data for machine learning in diabetes care

TL;DR

This study shows how preprocessing real-world diabetes data affects machine learning models, finding that worse baseline health predicts better HbA1c improvement.

Contribution

The study evaluates the impact of different preprocessing pipelines on ML models for diabetes care using real-world data.

Findings

56% of patients had a HbA1c decrease after three years, with higher baseline HbA1c and BMI.

ML model performance was stable across preprocessing methods, with DTC benefiting from missing data imputation.

Worse baseline values predicted HbA1c improvement, confirmed by both EDA and SHAP explanations.

Abstract

Machine Learning (ML) applied to healthcare Real World Data (RWD) may improve patient management. RWD, however, requires extensive preprocessing to make it ML-ready. Our aim was to explore the impact of preprocessing on ML models applied to RWD from 20 years of type 2 diabetes patients visits.

Our cohort consisted of patients with at least two glycated hemoglobin (HbA1c) measurements three years apart. We set up three different experimental settings consisting of different data preprocessing pipelines. Logistic Regression (LR), XGBoost and a Decision Tree Classifier (DTC) were then applied and tuned to optimize precision.

The final dataset comprised 12 variables from 1,651 patients treated between 2003 and 2023. 921 (56%) patients had a HbA1c decrease at three years. This group had a higher baseline HbA1c, higher BMI and shorter first visit gap from the date of diagnosis (p < 0.0001). Precision scores for LR, XGBoost did not vary across different experimental conditions while DTC benefitted from missing data imputation. Shapley Additive Explanations confirmed the Exploratory Data Analysis findings, with worse baseline values being predictors of HbA1c decrease at three years.

ML models' performance and their explanation did not vary substantially across experimental conditions, with worse baseline values being predictors of HbA1c decrease at three years. Insights such as this, extracted by ML application to RWD, enable clinical discussion and may foster improvements in patient management.

Figures