Hybrid CNN-Transformer Architecture for Arabic Speech Emotion Recognition

TL;DR

This paper introduces a hybrid CNN-Transformer model for Arabic speech emotion recognition, achieving high accuracy on the EYASE corpus by combining spectral feature extraction with attention mechanisms.

Contribution

It presents a novel CNN-Transformer architecture tailored for Arabic SER, addressing the scarcity of annotated datasets and demonstrating superior performance.

Findings

Achieved 97.8% accuracy on EYASE corpus.

Macro F1-score of 0.98 indicates high classification performance.

Hybrid model effectively captures spectral features and temporal dependencies.

Abstract

Recognizing emotions from speech using machine learning has become an active research area due to its importance in building human-centered applications. However, while many studies have been conducted in English, German, and other European and Asian languages, research in Arabic remains scarce because of the limited availability of annotated datasets. In this paper, we present an Arabic Speech Emotion Recognition (SER) system based on a hybrid CNN-Transformer architecture. The model leverages convolutional layers to extract discriminative spectral features from Mel-spectrogram inputs and Transformer encoders to capture long-range temporal dependencies in speech. Experiments were conducted on the EYASE (Egyptian Arabic speech emotion) corpus, and the proposed model achieved 97.8% accuracy and a macro F1-score of 0.98. These results demonstrate the effectiveness of combining convolutional feature extraction with attention-based modeling for Arabic SER and highlight the potential of Transformer-based approaches in low-resource languages.