DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized Patients

TL;DR

This paper presents DRG-LLaMA, a fine-tuned large language model that significantly improves the accuracy and efficiency of predicting Diagnosis-Related Groups for hospitalized patients using clinical notes.

Contribution

Introduces DRG-LLaMA, a LLaMA-based model fine-tuned with LoRA on clinical data, outperforming previous models in DRG prediction tasks.

Findings

DRG-LLaMA achieved a macro F1 score of 0.327.

Top-1 accuracy for DRG prediction was 52.0%.

Model performance improved with larger parameters and longer input context.

Abstract

In the U.S. inpatient payment system, the Diagnosis-Related Group (DRG) is pivotal, but its assignment process is inefficient. The study introduces DRG-LLaMA, an advanced large language model (LLM) fine-tuned on clinical notes to enhance DRGs assignment. Utilizing LLaMA as the foundational model and optimizing it through Low-Rank Adaptation (LoRA) on 236,192 MIMIC-IV discharge summaries, our DRG-LLaMA-7B model exhibited a noteworthy macro-averaged F1 score of 0.327, a top-1 prediction accuracy of 52.0%, and a macro-averaged Area Under the Curve (AUC) of 0.986, with a maximum input token length of 512. This model surpassed the performance of prior leading models in DRG prediction, showing a relative improvement of 40.3% and 35.7% in macro-averaged F1 score compared to ClinicalBERT and CAML, respectively. Applied to base DRG and complication or comorbidity (CC)/major complication or comorbidity (MCC) prediction, DRG-LLaMA achieved a top-1 prediction accuracy of 67.8% and 67.5%, respectively. Additionally, our findings indicate that DRG-LLaMA's performance correlates with increased model parameters and input context lengths.