Design, construction and evaluation of emotional multimodal pathological speech database

TL;DR

This paper presents the creation and evaluation of the first Chinese multimodal emotional pathological speech database, aiding research on emotion expression in dysarthria patients, with promising recognition accuracy results.

Contribution

It introduces a novel Chinese emotional pathological speech database with multi-perspective data and labels, facilitating emotion recognition research in dysarthria.

Findings

Automatic emotion recognition accuracy of 78% for controls in speech.

Recognition accuracy of 60% for patients in speech.

Disease impacts emotional expression as shown by recognition results.

Abstract

The lack of an available emotion pathology database is one of the key obstacles in studying the emotion expression status of patients with dysarthria. The first Chinese multimodal emotional pathological speech database containing multi-perspective information is constructed in this paper. It includes 29 controls and 39 patients with different degrees of motor dysarthria, expressing happy, sad, angry and neutral emotions. All emotional speech was labeled for intelligibility, types and discrete dimensional emotions by developed WeChat mini-program. The subjective analysis justifies from emotion discrimination accuracy, speech intelligibility, valence-arousal spatial distribution, and correlation between SCL-90 and disease severity. The automatic recognition tested on speech and glottal data, with average accuracy of 78% for controls and 60% for patients in audio, while 51% for controls and 38% for patients in glottal data, indicating an influence of the disease on emotional expression.