Enhancing Antidiabetic Drug Selection Using Transformers: Machine-Learning Model Development

TL;DR

This study developed a machine learning model using transformers to predict diabetes drug prescriptions, aiming to help nonspecialists make better treatment decisions.

Contribution

A transformer-based model that accurately predicts antidiabetic drug prescriptions with high ROC-AUC performance.

Findings

The model achieved a microaverage ROC-AUC of 0.993 and a macroaverage ROC-AUC of 0.988, surpassing the target of 0.95.

The model outperformed LightGBM and achieved high accuracy for 43 out of 44 drugs.

Training on 5-year data yielded better performance than 10-year data, suggesting recent patterns are more predictive.

Abstract

Diabetes affects millions worldwide. Primary care physicians provide a significant portion of care, and they often struggle with selecting appropriate medications.

This study aimed to develop a model that accurately predicts what drug an endocrinologist would prescribe based on the current measurements. The goal was to create a system that would assist nonspecialists in choosing medications, thereby potentially improving diabetes treatment outcomes. Based on the performance of previous studies, we set a performance target of achieving a receiver operating characteristic area under the curve (ROC-AUC) above 0.95.

A transformer-based encoder-decoder model predicts whether 44 types of diabetes drugs will be prescribed. The model uses sequences of age, sex, history for 12 laboratory tests, and prescribed drug history as inputs. We assessed the model using the electronic health records from 7034 patients with diabetes seeing endocrinologists between 2012 and 2022 at the University of Tokyo Hospital. We assessed model performance trained on data subsets spanning different time periods (2, 5, and 10 years) using micro- and macro-averaged ROC-AUC on a hold-out test set comprising data solely from 2022. The model’s performance was compared against LightGBM.

The model trained on data from the past 5 years (2017‐2021) yielded the best predictive performance, achieving a microaverage (95% CI) ROC-AUC of 0.993 (0.992-0.994) and a macroaverage (95% CI) ROC-AUC of 0.988 (0.980-0.993). The model achieved an ROC-AUC above 0.95 for 43 out of 44 drugs. These results surpassed the predefined performance target and outperformed both previous studies and the LightGBM model’s microaverage ROC-AUC of 0.988 (0.985-0.990) in terms of prediction accuracy. Furthermore, training the model with short-term data from the past 5 years yielded high accuracy compared to using data from the past 10 years, suggesting that learning from more recent prescribing patterns might be advantageous.

The proposed model demonstrates the feasibility of accurately predicting the next prescribed drugs. This model, trained from the past prescriptions of endocrinologists, has the potential to provide information that can assist nonspecialists in making diabetes-treatment decisions. Future studies will focus on incorporating important factors such as prescription contraindications and constraints to enhance safety, as well as leveraging large-scale clinical data across multiple hospitals to improve the generalizability of the model.

Figures