Emotion Recognition of the Singing Voice: Toward a Real-Time Analysis Tool for Singers

TL;DR

This paper develops a new model for real-time emotion recognition in singing voices, addressing noise challenges and aiming to enhance biofeedback applications across various performance and therapeutic contexts.

Contribution

It introduces a novel approach that incorporates psycho-acoustic properties to improve emotion recognition in noisy singing data, advancing AI's understanding of emotional cues in diverse human signals.

Findings

Model effectively recognizes emotions in noisy singing data

Incorporates psycho-acoustic features for improved accuracy

Lays groundwork for multi-modal emotion recognition systems

Abstract

Current computational-emotion research has focused on applying acoustic properties to analyze how emotions are perceived mathematically or used in natural language processing machine learning models. While recent interest has focused on analyzing emotions from the spoken voice, little experimentation has been performed to discover how emotions are recognized in the singing voice -- both in noiseless and noisy data (i.e., data that is either inaccurate, difficult to interpret, has corrupted/distorted/nonsense information like actual noise sounds in this case, or has a low ratio of usable/unusable information). Not only does this ignore the challenges of training machine learning models on more subjective data and testing them with much noisier data, but there is also a clear disconnect in progress between advancing the development of convolutional neural networks and the goal of emotionally cognizant artificial intelligence. By training a new model to include this type of information with a rich comprehension of psycho-acoustic properties, not only can models be trained to recognize information within extremely noisy data, but advancement can be made toward more complex biofeedback applications -- including creating a model which could recognize emotions given any human information (language, breath, voice, body, posture) and be used in any performance medium (music, speech, acting) or psychological assistance for patients with disorders such as BPD, alexithymia, autism, among others. This paper seeks to reflect and expand upon the findings of related research and present a stepping-stone toward this end goal.