DepressLLM: Interpretable domain-adapted language model for depression detection from real-world narratives

TL;DR

DepressLLM is an interpretable language model trained on autobiographical narratives that improves depression detection accuracy and confidence estimation, demonstrating potential for early psychiatric diagnosis.

Contribution

Introduces DepressLLM with a novel dataset and the SToPS module for interpretable, high-performance depression prediction across diverse real-world data.

Findings

Achieved an AUC of 0.789, rising to 0.904 with high-confidence samples.

Validated robustness on multiple datasets including EMA and clinical interviews.

Identified key limitations through psychiatric review of misclassifications.

Abstract

Advances in large language models (LLMs) have enabled a wide range of applications. However, depression prediction is hindered by the lack of large-scale, high-quality, and rigorously annotated datasets. This study introduces DepressLLM, trained and evaluated on a novel corpus of 3,699 autobiographical narratives reflecting both happiness and distress. DepressLLM provides interpretable depression predictions and, via its Score-guided Token Probability Summation (SToPS) module, delivers both improved classification performance and reliable confidence estimates, achieving an AUC of 0.789, which rises to 0.904 on samples with confidence $\geq$ 0.95. To validate its robustness to heterogeneous data, we evaluated DepressLLM on in-house datasets, including an Ecological Momentary Assessment (EMA) corpus of daily stress and mood recordings, and on public clinical interview data. Finally, a psychiatric review of high-confidence misclassifications highlighted key model and data limitations that suggest directions for future refinements. These findings demonstrate that interpretable AI can enable earlier diagnosis of depression and underscore the promise of medical AI in psychiatry.