CheX-GPT: Harnessing Large Language Models for Enhanced Chest X-ray Report Labeling

TL;DR

This paper introduces CheX-GPT, a novel approach that leverages large language models for accurate, efficient, and scalable labeling of chest X-ray reports, and provides a new benchmark dataset for evaluation.

Contribution

It demonstrates GPT's effectiveness as a labeler, trains a faster BERT-based labeler using GPT-labeled data, and introduces the MIMIC-500 dataset for benchmarking.

Findings

CheX-GPT outperforms existing models in labeling accuracy.

CheX-GPT is more efficient and scalable than GPT-based labeling.

The MIMIC-500 dataset enables robust benchmarking of labelers.

Abstract

Free-text radiology reports present a rich data source for various medical tasks, but effectively labeling these texts remains challenging. Traditional rule-based labeling methods fall short of capturing the nuances of diverse free-text patterns. Moreover, models using expert-annotated data are limited by data scarcity and pre-defined classes, impacting their performance, flexibility and scalability. To address these issues, our study offers three main contributions: 1) We demonstrate the potential of GPT as an adept labeler using carefully designed prompts. 2) Utilizing only the data labeled by GPT, we trained a BERT-based labeler, CheX-GPT, which operates faster and more efficiently than its GPT counterpart. 3) To benchmark labeler performance, we introduced a publicly available expert-annotated test set, MIMIC-500, comprising 500 cases from the MIMIC validation set. Our findings demonstrate that CheX-GPT not only excels in labeling accuracy over existing models, but also showcases superior efficiency, flexibility, and scalability, supported by our introduction of the MIMIC-500 dataset for robust benchmarking. Code and models are available at https://github.com/Soombit-ai/CheXGPT.