A Multimodal Approach for Automatic Mania Assessment in Bipolar Disorder

TL;DR

This paper presents a multimodal system combining acoustic, linguistic, and visual data to automatically assess mania in bipolar disorder patients, achieving state-of-the-art accuracy and aiding remote diagnosis.

Contribution

It introduces a novel multimodal decision system for bipolar disorder assessment that outperforms previous methods on the Bipolar Disorder corpus.

Findings

Achieved 64.8% unweighted average recall score

Demonstrated the effectiveness of multimodal fusion techniques

Improved state-of-the-art performance on the dataset

Abstract

Bipolar disorder is a mental health disorder that causes mood swings that range from depression to mania. Diagnosis of bipolar disorder is usually done based on patient interviews, and reports obtained from the caregivers of the patients. Subsequently, the diagnosis depends on the experience of the expert, and it is possible to have confusions of the disorder with other mental disorders. Automated processes in the diagnosis of bipolar disorder can help providing quantitative indicators, and allow easier observations of the patients for longer periods. Furthermore, the need for remote treatment and diagnosis became especially important during the COVID-19 pandemic. In this thesis, we create a multimodal decision system based on recordings of the patient in acoustic, linguistic, and visual modalities. The system is trained on the Bipolar Disorder corpus. Comprehensive analysis of unimodal and multimodal systems, as well as various fusion techniques are performed. Besides processing entire patient sessions using unimodal features, a task-level investigation of the clips is studied. Using acoustic, linguistic, and visual features in a multimodal fusion system, we achieved a 64.8% unweighted average recall score, which improves the state-of-the-art performance achieved on this dataset.