Improving Speaker-independent Speech Emotion Recognition Using Dynamic Joint Distribution Adaptation

TL;DR

This paper introduces a Dynamic Joint Distribution Adaptation method to improve speaker-independent speech emotion recognition by reducing speaker bias and better handling multi-domain distribution shifts, resulting in superior performance.

Contribution

The paper proposes a novel DJDA approach that dynamically balances marginal and conditional distribution adaptation for speaker-invariant emotion recognition.

Findings

DJDA outperforms state-of-the-art methods in experiments.

Effective reduction of speaker bias in emotion features.

Dynamic balancing improves adaptation to new speakers.

Abstract

In speaker-independent speech emotion recognition, the training and testing samples are collected from diverse speakers, leading to a multi-domain shift challenge across the feature distributions of data from different speakers. Consequently, when the trained model is confronted with data from new speakers, its performance tends to degrade. To address the issue, we propose a Dynamic Joint Distribution Adaptation (DJDA) method under the framework of multi-source domain adaptation. DJDA firstly utilizes joint distribution adaptation (JDA), involving marginal distribution adaptation (MDA) and conditional distribution adaptation (CDA), to more precisely measure the multi-domain distribution shifts caused by different speakers. This helps eliminate speaker bias in emotion features, allowing for learning discriminative and speaker-invariant speech emotion features from coarse-level to fine-level. Furthermore, we quantify the adaptation contributions of MDA and CDA within JDA by using a dynamic balance factor based on $\mathcal{A}$-Distance, promoting to effectively handle the unknown distributions encountered in data from new speakers. Experimental results demonstrate the superior performance of our DJDA as compared to other state-of-the-art (SOTA) methods.