Deep Architecture Enhancing Robustness to Noise, Adversarial Attacks, and Cross-corpus Setting for Speech Emotion Recognition

TL;DR

This paper introduces a deep neural network architecture combining DenseNet, LSTM, and Highway Networks, along with data augmentation, to improve robustness of speech emotion recognition systems against noise, adversarial attacks, and cross-corpus variations.

Contribution

The study presents a novel deep architecture and data augmentation strategy that enhance the robustness of speech emotion recognition to various data shifts and attacks.

Findings

Improved accuracy under noisy conditions

Enhanced resistance to adversarial attacks

Effective cross-corpus generalization

Abstract

Speech emotion recognition systems (SER) can achieve high accuracy when the training and test data are identically distributed, but this assumption is frequently violated in practice and the performance of SER systems plummet against unforeseen data shifts. The design of robust models for accurate SER is challenging, which limits its use in practical applications. In this paper we propose a deeper neural network architecture wherein we fuse DenseNet, LSTM and Highway Network to learn powerful discriminative features which are robust to noise. We also propose data augmentation with our network architecture to further improve the robustness. We comprehensively evaluate the architecture coupled with data augmentation against (1) noise, (2) adversarial attacks and (3) cross-corpus settings. Our evaluations on the widely used IEMOCAP and MSP-IMPROV datasets show promising results when compared with existing studies and state-of-the-art models.