Leveraging LLMs for Structured Data Extraction from Unstructured Patient Records

TL;DR

This paper introduces a secure, scalable framework using locally deployed large language models to automate extraction of structured data from unstructured clinical notes, aiming to reduce manual review effort and improve data consistency in clinical research.

Contribution

It presents a novel modular system integrating retrieval augmented generation with LLMs for accurate, secure, and scalable extraction of clinical features from unstructured EHR narratives.

Findings

Achieved high accuracy in extracting medical features from patient notes.

Identified annotation errors missed in manual review.

Demonstrated potential to reduce manual chart review workload.

Abstract

Manual chart review remains an extremely time-consuming and resource-intensive component of clinical research, requiring experts to extract often complex information from unstructured electronic health record (EHR) narratives. We present a secure, modular framework for automated structured feature extraction from clinical notes leveraging locally deployed large language models (LLMs) on institutionally approved, Health Insurance Portability and Accountability Act (HIPPA)-compliant compute infrastructure. This system integrates retrieval augmented generation (RAG) and structured response methods of LLMs into a widely deployable and scalable container to provide feature extraction for diverse clinical domains. In evaluation, the framework achieved high accuracy across multiple medical characteristics present in large bodies of patient notes when compared against an expert-annotated dataset and identified several annotation errors missed in manual review. This framework demonstrates the potential of LLM systems to reduce the burden of manual chart review through automated extraction and increase consistency in data capture, accelerating clinical research.