Deep Residual Local Feature Learning for Speech Emotion Recognition

TL;DR

This paper introduces DeepResLFLB, a deep residual local feature learning block, to enhance speech emotion recognition by addressing vanishing gradient issues and improving accuracy on standard datasets.

Contribution

It proposes a novel deep residual local feature learning block (DeepResLFLB) combining cascade blocks to improve SER performance and training efficiency.

Findings

Significant improvements in accuracy, precision, recall, and F1-score on EMODB and RAVDESS datasets.

Effective mitigation of vanishing gradient problem in deep SER models.

Enhanced local and hierarchical feature learning for speech emotion recognition.

Abstract

Speech Emotion Recognition (SER) is becoming a key role in global business today to improve service efficiency, like call center services. Recent SERs were based on a deep learning approach. However, the efficiency of deep learning depends on the number of layers, i.e., the deeper layers, the higher efficiency. On the other hand, the deeper layers are causes of a vanishing gradient problem, a low learning rate, and high time-consuming. Therefore, this paper proposed a redesign of existing local feature learning block (LFLB). The new design is called a deep residual local feature learning block (DeepResLFLB). DeepResLFLB consists of three cascade blocks: LFLB, residual local feature learning block (ResLFLB), and multilayer perceptron (MLP). LFLB is built for learning local correlations along with extracting hierarchical correlations; DeepResLFLB can take advantage of repeatedly learning to explain more detail in deeper layers using residual learning for solving vanishing gradient and reducing overfitting; and MLP is adopted to find the relationship of learning and discover probability for predicted speech emotions and gender types. Based on two available published datasets: EMODB and RAVDESS, the proposed DeepResLFLB can significantly improve performance when evaluated by standard metrics: accuracy, precision, recall, and F1-score.