In-context learning capabilities of Large Language Models to detect suicide risk among adolescents from speech transcripts

TL;DR

This study demonstrates that large language models can effectively detect suicide risk in adolescents using speech transcripts, outperforming traditional methods and highlighting the potential of LLMs in mental health screening.

Contribution

The paper introduces a novel in-context learning approach with LLMs for suicide risk detection from speech transcripts, achieving high accuracy without fine-tuning.

Findings

LLMs achieved 0.68 accuracy using transcripts alone.

Prompt example augmentation improved performance significantly.

The approach outperformed traditional fine-tuning methods.

Abstract

Early suicide risk detection in adolescents is critical yet hindered by scalability challenges of current assessments. This paper presents our approach to the first SpeechWellness Challenge (SW1), which aims to assess suicide risk in Chinese adolescents through speech analysis. Due to speech anonymization constraints, we focused on linguistic features, leveraging Large Language Models (LLMs) for transcript-based classification. Using DSPy for systematic prompt engineering, we developed a robust in-context learning approach that outperformed traditional fine-tuning on both linguistic and acoustic markers. Our systems achieved third and fourth places among 180+ submissions, with 0.68 accuracy (F1=0.7) using only transcripts. Ablation analyses showed that increasing prompt example improved performance (p=0.003), with varying effects across model types and sizes. These findings advance automated suicide risk assessment and demonstrate LLMs' value in mental health applications.