THAI Speech Emotion Recognition (THAI-SER) corpus

TL;DR

The paper introduces THAI-SER, a comprehensive Thai speech emotion recognition corpus with diverse recordings, annotations, and quality controls, facilitating research in Thai emotion recognition.

Contribution

It presents the first large-scale Thai speech emotion corpus with rigorous annotation and quality control, enabling improved emotion recognition models for Thai language.

Findings

Achieved an inter-annotator reliability score of 0.692

Human recognition accuracy reached 0.772 after filtering

Model trained on the corpus performs well in in-corpus and cross-corpus evaluations

Abstract

We present the first sizeable corpus of Thai speech emotion recognition, THAI-SER, containing 41 hours and 36 minutes (27,854 utterances) from 100 recordings made in different recording environments: Zoom and two studio setups. The recordings contain both scripted and improvised sessions, acted by 200 professional actors (112 females and 88 males, aged 18 to 55) and were directed by professional directors. There are five primary emotions: neutral, angry, happy, sad, and frustrated, assigned to the actors when recording utterances. The utterances are annotated with an emotional category using crowdsourcing. To control the annotation process's quality, we also design an extensive filtering and quality control scheme to ensure that the majority agreement score remains above 0.71. We evaluate our annotated corpus using two metrics: inter-annotator reliability and human recognition accuracy. Inter-annotator reliability score was calculated using Krippendorff's alpha, where our corpus, after filtering, achieved an alpha score of 0.692, higher than a recommendation of 0.667. For human recognition accuracy, our corpus scored up to 0.772 post-filtering. We also provide the results of the model trained on the corpus evaluated on both in-corpus and cross-corpus setups. The corpus is publicly available under a Creative Commons BY-SA 4.0, as well as our codes for the experiments.