Speaker Attentive Speech Emotion Recognition

TL;DR

This paper introduces a speaker-attentive speech emotion recognition system that leverages speaker identity information through a novel Self Speaker Attention mechanism, significantly improving emotion recognition accuracy.

Contribution

The work proposes a dual-classifier system with SSA that enhances emotion recognition by incorporating speaker identity, achieving state-of-the-art results.

Findings

Improved emotion recognition accuracy on Att-HACK and IEMOCAP datasets.

SSA mechanism effectively focuses on emotional speech features.

Achieved state-of-the-art unweighted average recall scores.

Abstract

Speech Emotion Recognition (SER) task has known significant improvements over the last years with the advent of Deep Neural Networks (DNNs). However, even the most successful methods are still rather failing when adaptation to specific speakers and scenarios is needed, inevitably leading to poorer performances when compared to humans. In this paper, we present novel work based on the idea of teaching the emotion recognition network about speaker identity. Our system is a combination of two ACRNN classifiers respectively dedicated to speaker and emotion recognition. The first informs the latter through a Self Speaker Attention (SSA) mechanism that is shown to considerably help to focus on emotional information of the speech signal. Experiments on social attitudes database Att-HACK and IEMOCAP corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.