Effect of different splitting criteria on the performance of speech emotion recognition

TL;DR

This study investigates how different data splitting criteria affect speech emotion recognition performance, revealing that more challenging splits lead to degraded accuracy and highlighting the complexity of emotion recognition across linguistic variations.

Contribution

It systematically compares various splitting criteria in SER, emphasizing the impact of sentence independence on recognition accuracy and revealing the relative difficulty of different data splits.

Findings

Sentence-open criteria degrade SER performance.

Text-independent splits are more challenging than speaker-independent splits.

Recognition difficulty order: text-independent > speaker+text-independent > speaker-independent > speaker+text-dependent.

Abstract

Traditional speech emotion recognition (SER) evaluations have been performed merely on a speaker-independent condition; some of them even did not evaluate their result on this condition. This paper highlights the importance of splitting training and test data for SER by script, known as sentence-open or text-independent criteria. The results show that employing sentence-open criteria degraded the performance of SER. This finding implies the difficulties of recognizing emotion from speech in different linguistic information embedded in acoustic information. Surprisingly, text-independent criteria consistently performed worse than speaker+text-independent criteria. The full order of difficulties for splitting criteria on SER performances from the most difficult to the easiest is text-independent, speaker+text-independent, speaker-independent, and speaker+text-dependent. The gap between speaker+text-independent and text-independent was smaller than other criteria, strengthening the difficulties of recognizing emotion from speech in different sentences.