SERAB: A multi-lingual benchmark for speech emotion recognition

TL;DR

SERAB is a comprehensive benchmark framework that evaluates speech emotion recognition models across multiple languages and datasets, helping to standardize comparisons and assess generalization of different approaches.

Contribution

The paper introduces SERAB, a multi-lingual benchmark with a standardized evaluation protocol for speech emotion recognition, addressing inconsistencies in dataset use and evaluation methods.

Findings

Using only a subset of data can bias evaluation results.

Compliance with the protocol improves the reliability of model comparisons.

State-of-the-art DNN representations outperform traditional features.

Abstract

Recent developments in speech emotion recognition (SER) often leverage deep neural networks (DNNs). Comparing and benchmarking different DNN models can often be tedious due to the use of different datasets and evaluation protocols. To facilitate the process, here, we present the Speech Emotion Recognition Adaptation Benchmark (SERAB), a framework for evaluating the performance and generalization capacity of different approaches for utterance-level SER. The benchmark is composed of nine datasets for SER in six languages. Since the datasets have different sizes and numbers of emotional classes, the proposed setup is particularly suitable for estimating the generalization capacity of pre-trained DNN-based feature extractors. We used the proposed framework to evaluate a selection of standard hand-crafted feature sets and state-of-the-art DNN representations. The results highlight that using only a subset of the data included in SERAB can result in biased evaluation, while compliance with the proposed protocol can circumvent this issue.