Clinically Inspired Symptom-Guided Depression Detection from Emotion-Aware Speech Representations

TL;DR

This paper introduces a symptom-guided, emotion-aware speech analysis framework for depression severity estimation, improving interpretability and performance by aligning speech segments with specific symptoms using a cross-attention mechanism.

Contribution

It presents a novel symptom-specific, clinically inspired approach that models depression symptoms explicitly from speech using a cross-attention mechanism and learnable parameters.

Findings

Outperforms prior methods on the EDAIC dataset

Attention focuses on speech segments related to multiple symptoms

Enhanced interpretability of depression cues in speech

Abstract

Depression manifests through a diverse set of symptoms such as sleep disturbance, loss of interest, and concentration difficulties. However, most existing works treat depression prediction either as a binary label or an overall severity score without explicitly modeling symptom-specific information. This limits their ability to provide symptom-level analysis relevant to clinical screening. To address this, we propose a symptom-specific and clinically inspired framework for depression severity estimation from speech. Our approach uses a symptom-guided cross-attention mechanism that aligns PHQ-8 questionnaire items with emotion-aware speech representations to identify which segments of a participant's speech are more important to each symptom. To account for differences in how symptoms are expressed over time, we introduce a learnable symptom-specific parameter that adaptively controls the sharpness of attention distributions. Our results on EDAIC, a standard clinical-style dataset, demonstrate improved performance outperforming prior works. Further, analyzing the attention distributions showed that higher attention is assigned to utterances containing cues related to multiple depressive symptoms, highlighting the interpretability of our approach. These findings outline the importance of symptom-guided and emotion-aware modeling for speech-based depression screening.