Unrequited Emotions: Investigating the Gaps in Motivation and Practice in Speech Emotion Recognition Research

TL;DR

This paper critically examines speech emotion recognition research, revealing a disconnect between stated motivations and actual research practices, which raises ethical concerns about potential misuse and downstream impacts.

Contribution

It systematically surveys SER research to identify gaps between motivations and datasets, emphasizing the need for alignment with real-world use cases to address ethical issues.

Findings

SER research motivations often do not match datasets used

Common datasets do not reflect proposed deployment contexts

Gaps may lead to ethical concerns and misuse

Abstract

Critical analyses of emotion recognition technology have raised ethical concerns around task validity and potential downstream impacts, urging researchers to ensure alignment between their stated motivations and practice. However, these discussions have not adequately influenced or drawn from research on speech emotion recognition (SER). We address this gap by conducting a systematic survey of SER research to uncover what stated motivations drive this work and if they align with the datasets and emotions studied. We find that while SER research identifies appealing goals, such as well-situated voice-activated systems or healthcare applications, commonly-used datasets do not reflect these proposed deployment contexts, thus presenting a gap between motivations and research practices. We argue that such gaps engender ethical concerns, and that SER research should reassert itself with concrete use-cases to prevent misinterpretations, misuse, and downstream harms.