Speech-Based Emotion Recognition using Neural Networks and Information Visualization

TL;DR

This paper presents a machine learning-based tool that classifies emotions from speech samples and visualizes the results in an intuitive dashboard tailored for therapists, aiding in health assessment.

Contribution

It introduces a user-friendly visualization system for speech emotion recognition tailored to clinical therapy settings, addressing the gap in interpretability and usability.

Findings

Effective emotion classification from speech samples.

Therapists can gain insights through the visualization dashboard.

Potential to improve emotional assessment in health care.

Abstract

Emotions recognition is commonly employed for health assessment. However, the typical metric for evaluation in therapy is based on patient-doctor appraisal. This process can fall into the issue of subjectivity, while also requiring healthcare professionals to deal with copious amounts of information. Thus, machine learning algorithms can be a useful tool for the classification of emotions. While several models have been developed in this domain, there is a lack of userfriendly representations of the emotion classification systems for therapy. We propose a tool which enables users to take speech samples and identify a range of emotions (happy, sad, angry, surprised, neutral, clam, disgust, and fear) from audio elements through a machine learning model. The dashboard is designed based on local therapists' needs for intuitive representations of speech data in order to gain insights and informative analyses of their sessions with their patients.