Cross-Lingual Cross-Age Group Adaptation for Low-Resource Elderly Speech Emotion Recognition

TL;DR

This paper investigates cross-lingual and cross-age group transferability in speech emotion recognition, highlighting the importance of specific speech features and data augmentation for improving model performance across diverse demographics.

Contribution

It introduces new datasets for elderly speech emotion recognition in English, Mandarin, and Cantonese, and analyzes the effects of linguistic distance and data augmentation on transferability.

Findings

Cross-lingual inference is generally unsuitable due to feature differences.

Cross-group data augmentation improves model regularization.

Linguistic distance significantly affects transferability.

Abstract

Speech emotion recognition plays a crucial role in human-computer interactions. However, most speech emotion recognition research is biased toward English-speaking adults, which hinders its applicability to other demographic groups in different languages and age groups. In this work, we analyze the transferability of emotion recognition across three different languages--English, Mandarin Chinese, and Cantonese; and 2 different age groups--adults and the elderly. To conduct the experiment, we develop an English-Mandarin speech emotion benchmark for adults and the elderly, BiMotion, and a Cantonese speech emotion dataset, YueMotion. This study concludes that different language and age groups require specific speech features, thus making cross-lingual inference an unsuitable method. However, cross-group data augmentation is still beneficial to regularize the model, with linguistic distance being a significant influence on cross-lingual transferability. We release publicly release our code at https://github.com/HLTCHKUST/elderly_ser.