EHRmonize: A Framework for Medical Concept Abstraction from Electronic Health Records using Large Language Models

TL;DR

EHRmonize is a framework that uses large language models to efficiently abstract medical concepts from electronic health records, significantly reducing annotation time and aiding clinical data processing.

Contribution

The paper introduces EHRmonize, a novel framework leveraging LLMs for medical concept abstraction from EHRs, demonstrating high accuracy and efficiency improvements.

Findings

GPT-4o achieved 97% accuracy in route name identification

82% accuracy in generic drug name extraction

100% accuracy in binary classification of antibiotics

Abstract

Electronic health records (EHRs) contain vast amounts of complex data, but harmonizing and processing this information remains a challenging and costly task requiring significant clinical expertise. While large language models (LLMs) have shown promise in various healthcare applications, their potential for abstracting medical concepts from EHRs remains largely unexplored. We introduce EHRmonize, a framework leveraging LLMs to abstract medical concepts from EHR data. Our study uses medication data from two real-world EHR databases to evaluate five LLMs on two free-text extraction and six binary classification tasks across various prompting strategies. GPT-4o's with 10-shot prompting achieved the highest performance in all tasks, accompanied by Claude-3.5-Sonnet in a subset of tasks. GPT-4o achieved an accuracy of 97% in identifying generic route names, 82% for generic drug names, and 100% in performing binary classification of antibiotics. While EHRmonize significantly enhances efficiency, reducing annotation time by an estimated 60%, we emphasize that clinician oversight remains essential. Our framework, available as a Python package, offers a promising tool to assist clinicians in EHR data abstraction, potentially accelerating healthcare research and improving data harmonization processes.