Speech Emotion Recognition with Global-Aware Fusion on Multi-scale Feature Representation

TL;DR

This paper introduces a novel neural network architecture called GLAM that enhances speech emotion recognition by capturing multi-scale features and global emotional information, outperforming previous methods on benchmark data.

Contribution

The paper proposes a global-aware fusion module within a multi-scale CNN framework for improved speech emotion recognition, addressing limitations of local attention mechanisms.

Findings

Achieved 2.5% to 4.5% improvements on IEMOCAP benchmark

Effectively captures multi-scale emotional features

Demonstrates superiority over state-of-the-art methods

Abstract

Speech Emotion Recognition (SER) is a fundamental task to predict the emotion label from speech data. Recent works mostly focus on using convolutional neural networks~(CNNs) to learn local attention map on fixed-scale feature representation by viewing time-varied spectral features as images. However, rich emotional feature at different scales and important global information are not able to be well captured due to the limits of existing CNNs for SER. In this paper, we propose a novel GLobal-Aware Multi-scale (GLAM) neural network (The code is available at https://github.com/lixiangucas01/GLAM) to learn multi-scale feature representation with global-aware fusion module to attend emotional information. Specifically, GLAM iteratively utilizes multiple convolutional kernels with different scales to learn multiple feature representation. Then, instead of using attention-based methods, a simple but effective global-aware fusion module is applied to grab most important emotional information globally. Experiments on the benchmark corpus IEMOCAP over four emotions demonstrates the superiority of our proposed model with 2.5% to 4.5% improvements on four common metrics compared to previous state-of-the-art approaches.