CNN+LSTM Architecture for Speech Emotion Recognition with Data Augmentation

TL;DR

This paper introduces a CNN+LSTM neural network architecture for speech emotion recognition, utilizing data augmentation and advanced training techniques to achieve competitive accuracy on the IEMOCAP dataset.

Contribution

It presents a novel combination of convolutional and recurrent layers with data augmentation methods for improved speech emotion recognition.

Findings

Achieved 64.5% weighted accuracy on IEMOCAP

Achieved 61.7% unweighted accuracy on IEMOCAP

Demonstrated effectiveness of data augmentation and optimizer adjustments

Abstract

In this work we design a neural network for recognizing emotions in speech, using the IEMOCAP dataset. Following the latest advances in audio analysis, we use an architecture involving both convolutional layers, for extracting high-level features from raw spectrograms, and recurrent ones for aggregating long-term dependencies. We examine the techniques of data augmentation with vocal track length perturbation, layer-wise optimizer adjustment, batch normalization of recurrent layers and obtain highly competitive results of 64.5% for weighted accuracy and 61.7% for unweighted accuracy on four emotions.