Multi-modal Depression Estimation based on Sub-attentional Fusion

TL;DR

This paper introduces a novel multi-modal depression estimation method using a sub-attention mechanism and convolutional bidirectional LSTM, achieving high accuracy and efficiency on the DAIC-WOZ benchmark.

Contribution

It proposes a new sub-attention based fusion approach for multi-modal depression detection, outperforming traditional methods and reducing preprocessing complexity.

Findings

Achieved 0.89 precision and 0.70 F1-score in depression detection.

Obtained 4.92 MAE in depression severity estimation.

Outperformed conventional late fusion approaches.

Abstract

Failure to timely diagnose and effectively treat depression leads to over 280 million people suffering from this psychological disorder worldwide. The information cues of depression can be harvested from diverse heterogeneous resources, e.g., audio, visual, and textual data, raising demand for new effective multi-modal fusion approaches for automatic estimation. In this work, we tackle the task of automatically identifying depression from multi-modal data and introduce a sub-attention mechanism for linking heterogeneous information while leveraging Convolutional Bidirectional LSTM as our backbone. To validate this idea, we conduct extensive experiments on the public DAIC-WOZ benchmark for depression assessment featuring different evaluation modes and taking gender-specific biases into account. The proposed model yields effective results with 0.89 precision and 0.70 F1-score in detecting major depression and 4.92 MAE in estimating the severity. Our attention-based fusion module consistently outperforms conventional late fusion approaches and achieves competitive performance compared to the previously published depression estimation frameworks, while learning to diagnose the disorder end-to-end and relying on far fewer preprocessing steps.