Emotion Profile Refinery for Speech Emotion Classification

TL;DR

This paper introduces an emotion profile refinery method that uses soft, dynamic labels to better capture emotional nuances in speech, improving emotion classification accuracy.

Contribution

It proposes the emotion profile refinery (EPR), an iterative approach that refines soft labels for speech emotion classification, addressing emotional impurity issues.

Findings

Significant accuracy improvements on three emotion corpora.

EPR effectively captures subtle emotional cues.

Soft, dynamic labels outperform hard labels in classification.

Abstract

Human emotions are inherently ambiguous and impure. When designing systems to anticipate human emotions based on speech, the lack of emotional purity must be considered. However, most of the current methods for speech emotion classification rest on the consensus, e.g., one single hard label for an utterance. This labeling principle imposes challenges for system performance considering emotional impurity. In this paper, we recommend the use of emotional profiles (EPs), which provides a time series of segment-level soft labels to capture the subtle blends of emotional cues present across a specific speech utterance. We further propose the emotion profile refinery (EPR), an iterative procedure to update EPs. The EPR method produces soft, dynamically-generated, multiple probabilistic class labels during successive stages of refinement, which results in significant improvements in the model accuracy. Experiments on three well-known emotion corpora show noticeable gain using the proposed method.