Using reasoning LLMs to extract SDOH events from clinical notes

TL;DR

This paper explores prompt engineering strategies for extracting Social Determinants of Health (SDOH) events from clinical notes using reasoning large language models, achieving high accuracy with simpler implementation.

Contribution

It introduces a novel prompt-based approach with few-shot learning and self-consistency mechanisms for effective SDOH extraction using LLMs.

Findings

Achieved a micro-F1 score of 0.866 in SDOH event extraction.

Demonstrated competitive performance with less complex implementation than BERT-based models.

Showed that reasoning LLMs are effective for structured information extraction from clinical notes.

Abstract

Social Determinants of Health (SDOH) refer to environmental, behavioral, and social conditions that influence how individuals live, work, and age. SDOH have a significant impact on personal health outcomes, and their systematic identification and management can yield substantial improvements in patient care. However, SDOH information is predominantly captured in unstructured clinical notes within electronic health records, which limits its direct use as machine-readable entities. To address this issue, researchers have employed Natural Language Processing (NLP) techniques using pre-trained BERT-based models, demonstrating promising performance but requiring sophisticated implementation and extensive computational resources. In this study, we investigated prompt engineering strategies for extracting structured SDOH events utilizing LLMs with advanced reasoning capabilities. Our method consisted of four modules: 1) developing concise and descriptive prompts integrated with established guidelines, 2) applying few-shot learning with carefully curated examples, 3) using a self-consistency mechanism to ensure robust outputs, and 4) post-processing for quality control. Our approach achieved a micro-F1 score of 0.866, demonstrating competitive performance compared to the leading models. The results demonstrated that LLMs with reasoning capabilities are effective solutions for SDOH event extraction, offering both implementation simplicity and strong performance.