EmoSphere-SER: Enhancing Speech Emotion Recognition Through Spherical Representation with Auxiliary Classification

TL;DR

EmoSphere-SER introduces a spherical representation and auxiliary classification to enhance speech emotion recognition accuracy by better modeling VAD dimensions and dynamics.

Contribution

The paper presents a novel joint model that uses spherical VAD classification to guide emotion prediction, incorporating dynamic weighting and self-attention mechanisms.

Findings

Outperforms baseline methods in speech emotion recognition

Effectively models VAD dimensions using spherical coordinates

Improves prediction consistency with structured learning

Abstract

Speech emotion recognition predicts a speaker's emotional state from speech signals using discrete labels or continuous dimensions such as arousal, valence, and dominance (VAD). We propose EmoSphere-SER, a joint model that integrates spherical VAD region classification to guide VAD regression for improved emotion prediction. In our framework, VAD values are transformed into spherical coordinates that are divided into multiple spherical regions, and an auxiliary classification task predicts which spherical region each point belongs to, guiding the regression process. Additionally, we incorporate a dynamic weighting scheme and a style pooling layer with multi-head self-attention to capture spectral and temporal dynamics, further boosting performance. This combined training strategy reinforces structured learning and improves prediction consistency. Experimental results show that our approach exceeds baseline methods, confirming the validity of the proposed framework.