Who is Speaking or Who is Depressed? A Controlled Study of Speaker Leakage in Speech-Based Depression Detection

TL;DR

This paper examines whether speech-based depression detection models genuinely identify depression biomarkers or merely recognize speaker identity cues, revealing significant speaker leakage issues.

Contribution

It introduces a data-splitting strategy to control speaker overlap and demonstrates that current models rely heavily on speaker identity rather than depression-specific features.

Findings

Speaker overlap boosts model performance significantly.

Accuracy drops sharply on unseen speakers.

Even domain-adversarial models show a performance gap.

Abstract

This study investigates whether speech-based depression detection models learn depression-related acoustic biomarkers or instead rely on speaker identity cues. Using the DAIC-WOZ dataset, we propose a data-splitting strategy that controls speaker overlap between training and test sets while keeping the training size constant, and evaluate three models of varying complexity. Results show that speaker overlap significantly boosts performance, whereas accuracy drops sharply on unseen speakers. Even with a Domain-Adversarial Neural Network, a substantial performance gap remains. These findings indicate that depression-related features extracted by current speech models are highly entangled with speaker identity. Conventional evaluation protocols may therefore overestimate generalization and clinical utility, highlighting the need for strictly speaker-independent evaluation.