# Enhancing Substance Use Detection in Clinical Notes with Large Language Models

## TL;DR

This paper shows how large language models can accurately detect substance use in medical notes, improving detection for clinical use.

## Contribution

The novel contribution is a fine-tuned LLM, Llama-DrugDetector-70B, achieving high accuracy in detecting substance use from clinical notes.

## Key findings

- Llama-DrugDetector-70B achieved F1-scores ≥ 0.95 for most substances.
- The model scored F1=0.815 for opioid misuse and F1=0.917 for polysubstance use.
- LLMs significantly enhance detection of substance use in clinical notes.

## Abstract

Identifying substance use behaviors in electronic health records (EHRs) is challenging because critical details are often buried in unstructured notes that use varied terminology and negation, requiring careful contextual interpretation to distinguish relevant use from historical mentions or denials. Using MIMIC-III/IV discharge summaries, we created a large, annotated drug detection dataset to tackle this problem and support future systemic substance use surveillance. We then investigated the performance of multiple large language models (LLMs) for detecting eight substance use categories within this data. Evaluating models in zero-shot, few-shot, and fine-tuning configurations, we found that a fine-tuned model, Llama-DrugDetector-70B, outperformed others. It achieved near-perfect F1-scores (≥ 0.95) for most individual substances and strong scores for more complex tasks like prescription opioid misuse (F1=0.815) and polysubstance use (F1=0.917). These findings demonstrate that LLMs significantly enhance detection, showing promise for clinical decision support and research, although further work on scalability is warranted.