A Multi-Stage Large Language Model Framework for Extracting Suicide-Related Social Determinants of Health

TL;DR

This paper introduces a multi-stage large language model framework that improves the extraction of suicide-related social determinants of health from unstructured text, enhancing accuracy, explainability, and efficiency for early intervention.

Contribution

The paper presents a novel multi-stage LLM framework that outperforms existing models in extracting SDoH factors related to suicide, with improved explainability and reduced inference costs.

Findings

Enhanced SDoH extraction accuracy and context retrieval.

Fine-tuning smaller models achieves comparable performance.

Intermediate explanations improve model transparency.

Abstract

Background: Understanding social determinants of health (SDoH) factors contributing to suicide incidents is crucial for early intervention and prevention. However, data-driven approaches to this goal face challenges such as long-tailed factor distributions, analyzing pivotal stressors preceding suicide incidents, and limited model explainability. Methods: We present a multi-stage large language model framework to enhance SDoH factor extraction from unstructured text. Our approach was compared to other state-of-the-art language models (i.e., pre-trained BioBERT and GPT-3.5-turbo) and reasoning models (i.e., DeepSeek-R1). We also evaluated how the model's explanations help people annotate SDoH factors more quickly and accurately. The analysis included both automated comparisons and a pilot user study. Results: We show that our proposed framework demonstrated performance boosts in the overarching task of extracting SDoH factors and in the finer-grained tasks of retrieving relevant context. Additionally, we show that fine-tuning a smaller, task-specific model achieves comparable or better performance with reduced inference costs. The multi-stage design not only enhances extraction but also provides intermediate explanations, improving model explainability. Conclusions: Our approach improves both the accuracy and transparency of extracting suicide-related SDoH from unstructured texts. These advancements have the potential to support early identification of individuals at risk and inform more effective prevention strategies.