Learning Evidence of Depression Symptoms via Prompt Induction

TL;DR

This paper introduces Symptom Induction, a novel method that improves the detection of depression symptoms in text by generating interpretable guidelines, outperforming standard LLM approaches especially for rare symptoms.

Contribution

The paper presents Symptom Induction, a new approach that enhances symptom classification accuracy and generalizes across related mental health conditions.

Findings

SI achieves the best weighted F1 scores across models.

SI significantly improves detection of infrequent symptoms.

Induced guidelines generalize to other disorders like bipolar and eating disorders.

Abstract

Depression places substantial pressure on mental health services, and many people describe their experiences outside clinical settings in high-volume user-generated text (e.g., online forums and social media). Automatically identifying clinical symptom evidence in such text can therefore complement limited clinical capacity and scale to large populations. We address this need through sentence-level classification of 21 depression symptoms from the BDI-II questionnaire, using BDI-Sen, a dataset annotated for symptom relevance. This task is fine-grained and highly imbalanced, and we find that common LLM approaches (zero-shot, in-context learning, and fine-tuning) struggle to apply consistent relevance criteria for most symptoms. We propose Symptom Induction (SI), a novel approach which compresses labeled examples into short, interpretable guidelines that specify what counts as evidence for each symptom and uses these guidelines to condition classification. Across four LLM families and eight models, SI achieves the best overall weighted F1 on BDI-Sen, with especially large gains for infrequent symptoms. Cross-domain evaluation on an external dataset further shows that induced guidelines generalize across other diseases shared symptomatology (bipolar and eating disorders).