Emotional Speech Recognition with Pre-trained Deep Visual Models

TL;DR

This paper introduces a novel approach for emotional speech recognition by converting acoustic features into images and leveraging pre-trained visual models, achieving state-of-the-art results on a standard dataset.

Contribution

It presents a new methodology that uses transfer learning with visual deep models for speech emotion recognition, including a novel feature-to-image conversion and training paradigm.

Findings

Achieved state-of-the-art accuracy on Berlin EMO-DB dataset.

Demonstrated effectiveness of visual models in speech emotion recognition.

Validated the approach across seven emotion categories.

Abstract

In this paper, we propose a new methodology for emotional speech recognition using visual deep neural network models. We employ the transfer learning capabilities of the pre-trained computer vision deep models to have a mandate for the emotion recognition in speech task. In order to achieve that, we propose to use a composite set of acoustic features and a procedure to convert them into images. Besides, we present a training paradigm for these models taking into consideration the different characteristics between acoustic-based images and regular ones. In our experiments, we use the pre-trained VGG-16 model and test the overall methodology on the Berlin EMO-DB dataset for speaker-independent emotion recognition. We evaluate the proposed model on the full list of the seven emotions and the results set a new state-of-the-art.