Deep Learning based Emotion Recognition System Using Speech Features and Transcriptions

TL;DR

This paper introduces a deep learning-based speech emotion recognition system that combines speech features like MFCC and spectrogram with transcriptions to improve accuracy, demonstrating superior performance on benchmark datasets.

Contribution

It presents a novel multi-input neural network architecture that integrates speech features and transcriptions for enhanced emotion recognition accuracy.

Findings

Combined MFCC-Text CNN outperforms other models

Achieves higher accuracy than state-of-the-art methods

Effective use of both speech features and transcriptions

Abstract

This paper proposes a speech emotion recognition method based on speech features and speech transcriptions (text). Speech features such as Spectrogram and Mel-frequency Cepstral Coefficients (MFCC) help retain emotion-related low-level characteristics in speech whereas text helps capture semantic meaning, both of which help in different aspects of emotion detection. We experimented with several Deep Neural Network (DNN) architectures, which take in different combinations of speech features and text as inputs. The proposed network architectures achieve higher accuracies when compared to state-of-the-art methods on a benchmark dataset. The combined MFCC-Text Convolutional Neural Network (CNN) model proved to be the most accurate in recognizing emotions in IEMOCAP data.