Multimodal Biomarkers for Schizophrenia: Towards Individual Symptom Severity Estimation

TL;DR

This paper introduces a multimodal deep learning framework that estimates individual schizophrenia symptom severity from speech, video, and text data, aiming to improve diagnostic accuracy and support personalized treatment.

Contribution

It presents a novel multimodal approach combining speech, video, and text modalities for detailed symptom severity estimation in schizophrenia, moving beyond simple classification.

Findings

Improved accuracy in symptom severity estimation over unimodal models

Demonstrated robustness of the multimodal framework across modalities

Potential for scalable, objective mental health assessment tools

Abstract

Studies on schizophrenia assessments using deep learning typically treat it as a classification task to detect the presence or absence of the disorder, oversimplifying the condition and reducing its clinical applicability. This traditional approach overlooks the complexity of schizophrenia, limiting its practical value in healthcare settings. This study shifts the focus to individual symptom severity estimation using a multimodal approach that integrates speech, video, and text inputs. We develop unimodal models for each modality and a multimodal framework to improve accuracy and robustness. By capturing a more detailed symptom profile, this approach can help in enhancing diagnostic precision and support personalized treatment, offering a scalable and objective tool for mental health assessment.