MIMIC-IV-Ext-PE: Using a large language model to predict pulmonary embolism phenotype in the MIMIC-IV dataset

TL;DR

This study leverages a large language model to automatically label pulmonary embolism in radiology reports within the MIMIC-IV dataset, enhancing data availability for research and demonstrating high accuracy compared to manual labels and diagnosis codes.

Contribution

It introduces VTE-BERT, a fine-tuned transformer model, for reliable automatic PE labeling in large clinical datasets, expanding research resources.

Findings

VTE-BERT achieved 92.4% sensitivity and 87.8% PPV in PE detection.

The model added nearly 20,000 labels to the dataset.

VTE-BERT outperformed diagnosis codes in accuracy.

Abstract

Pulmonary embolism (PE) is a leading cause of preventable in-hospital mortality. Advances in diagnosis, risk stratification, and prevention can improve outcomes. There are few large publicly available datasets that contain PE labels for research. Using the MIMIC-IV database, we extracted all available radiology reports of computed tomography pulmonary angiography (CTPA) scans and two physicians manually labeled the results as PE positive (acute PE) or PE negative. We then applied a previously finetuned Bio_ClinicalBERT transformer language model, VTE-BERT, to extract labels automatically. We verified VTE-BERT's reliability by measuring its performance against manual adjudication. We also compared the performance of VTE-BERT to diagnosis codes. We found that VTE-BERT has a sensitivity of 92.4% and positive predictive value (PPV) of 87.8% on all 19,942 patients with CTPA radiology reports from the emergency room and/or hospital admission. In contrast, diagnosis codes have a sensitivity of 95.4% and PPV of 83.8% on the subset of 11,990 hospitalized patients with discharge diagnosis codes. We successfully add nearly 20,000 labels to CTPAs in a publicly available dataset and demonstrate the external validity of a semi-supervised language model in accelerating hematologic research.