Leveraging Large Language Models to Extract Information on Substance Use Disorder Severity from Clinical Notes: A Zero-shot Learning Approach

TL;DR

This paper explores using large language models with zero-shot learning to extract detailed substance use disorder severity information from clinical notes, addressing limitations of traditional NLP methods and enhancing clinical decision-making.

Contribution

It introduces a workflow leveraging LLMs with prompts and post-processing to improve extraction of SUD severity from unstructured clinical text, demonstrating superior recall over rule-based methods.

Findings

LLMs outperform rule-based approaches in recall for SUD severity extraction

Effective extraction across 11 SUD categories from clinical notes

Demonstrates potential for improved risk assessment and treatment planning

Abstract

Substance use disorder (SUD) poses a major concern due to its detrimental effects on health and society. SUD identification and treatment depend on a variety of factors such as severity, co-determinants (e.g., withdrawal symptoms), and social determinants of health. Existing diagnostic coding systems used by American insurance providers, like the International Classification of Diseases (ICD-10), lack granularity for certain diagnoses, but clinicians will add this granularity (as that found within the Diagnostic and Statistical Manual of Mental Disorders classification or DSM-5) as supplemental unstructured text in clinical notes. Traditional natural language processing (NLP) methods face limitations in accurately parsing such diverse clinical language. Large Language Models (LLMs) offer promise in overcoming these challenges by adapting to diverse language patterns. This study investigates the application of LLMs for extracting severity-related information for various SUD diagnoses from clinical notes. We propose a workflow employing zero-shot learning of LLMs with carefully crafted prompts and post-processing techniques. Through experimentation with Flan-T5, an open-source LLM, we demonstrate its superior recall compared to the rule-based approach. Focusing on 11 categories of SUD diagnoses, we show the effectiveness of LLMs in extracting severity information, contributing to improved risk assessment and treatment planning for SUD patients.