Multi-Scale Temporal Transformer For Speech Emotion Recognition

TL;DR

This paper introduces a Multi-Scale Transformer model that enhances speech emotion recognition by capturing multi-scale local features, outperforming existing methods while reducing computational costs.

Contribution

The paper proposes a novel Multi-Scale Transformer with three components to improve local emotion feature learning and efficiency in speech emotion recognition.

Findings

Significantly outperforms vanilla Transformer and state-of-the-art methods

Effective in capturing multi-scale local emotion representations

Reduces computational cost compared to existing models

Abstract

Speech emotion recognition plays a crucial role in human-machine interaction systems. Recently various optimized Transformers have been successfully applied to speech emotion recognition. However, the existing Transformer architectures focus more on global information and require large computation. On the other hand, abundant speech emotional representations exist locally on different parts of the input speech. To tackle these problems, we propose a Multi-Scale TRansfomer (MSTR) for speech emotion recognition. It comprises of three main components: (1) a multi-scale temporal feature operator, (2) a fractal self-attention module, and (3) a scale mixer module. These three components can effectively enhance the transformer's ability to learn multi-scale local emotion representations. Experimental results demonstrate that the proposed MSTR model significantly outperforms a vanilla Transformer and other state-of-the-art methods across three speech emotion datasets: IEMOCAP, MELD and, CREMAD. In addition, it can greatly reduce the computational cost.