Depression diagnosis from patient interviews using multimodal machine learning

TL;DR

This study presents a multimodal machine learning approach that combines speech, language, and clinical data from patient interviews to improve the accuracy and utility of depression diagnosis, potentially aiding clinical decision-making.

Contribution

It introduces a novel multimodal framework that integrates multiple data sources for depression diagnosis, demonstrating improved accuracy and clinical utility over single-modality models.

Findings

Multimodal model achieved AUROC of 0.88.

Fused model showed good calibration and higher net clinical benefit.

Approach enhances early detection and supports clinical decision-making.

Abstract

Background: Depression is a major public health concern, affecting an estimated five percent of the global population. Early and accurate diagnosis is essential to initiate effective treatment, yet recognition remains challenging in many clinical contexts. Speech, language, and behavioral cues collected during patient interviews may provide objective markers that support clinical assessment. Methods: We developed a diagnostic approach that integrates features derived from patient interviews, including speech patterns, linguistic characteristics, and structured clinical information. Separate models were trained for each modality and subsequently combined through multimodal fusion to reflect the complexity of real-world psychiatric assessment. Model validity was assessed with established performance metrics, and further evaluated using calibration and decision-analytic approaches to estimate potential clinical utility. Results: The multimodal model achieved superior diagnostic accuracy compared to single-modality models, with an AUROC of 0.88 and a macro F1-score of 0.75. Importantly, the fused model demonstrated good calibration and offered higher net clinical benefit compared to baseline strategies, highlighting its potential to assist clinicians in identifying patients with depression more reliably. Conclusion: Multimodal analysis of patient interviews using machine learning may serve as a valuable adjunct to psychiatric evaluation. By combining speech, language, and clinical features, this approach provides a robust framework that could enhance early detection of depressive disorders and support evidence-based decision-making in mental healthcare.