Speaker Normalization for Self-supervised Speech Emotion Recognition

TL;DR

This paper introduces a gradient-based adversarial framework for speech emotion recognition that normalizes speaker traits to improve generalization, achieving state-of-the-art results on the IEMOCAP dataset.

Contribution

The paper proposes a novel adversarial learning approach to normalize speaker characteristics in speech emotion recognition models, enhancing their robustness.

Findings

State-of-the-art performance on IEMOCAP dataset

Effective normalization of speaker traits in emotion recognition

Improved generalization across speaker-dependent and independent settings

Abstract

Large speech emotion recognition datasets are hard to obtain, and small datasets may contain biases. Deep-net-based classifiers, in turn, are prone to exploit those biases and find shortcuts such as speaker characteristics. These shortcuts usually harm a model's ability to generalize. To address this challenge, we propose a gradient-based adversary learning framework that learns a speech emotion recognition task while normalizing speaker characteristics from the feature representation. We demonstrate the efficacy of our method on both speaker-independent and speaker-dependent settings and obtain new state-of-the-art results on the challenging IEMOCAP dataset.