A Multi-component CNN-RNN Approach for Dimensional Emotion Recognition in-the-wild

TL;DR

This paper introduces a multi-component CNN-RNN deep learning model for dimensional emotion recognition in videos, achieving improved accuracy on the OMG-Emotion Challenge dataset.

Contribution

It develops an extended CNN-RNN architecture that combines multiple features for better emotion dimension estimation in-the-wild videos.

Findings

Achieved best performance on valence and arousal estimation tasks.

Optimized architecture for the OMG-Emotion validation dataset.

Demonstrated effectiveness of multi-feature CNN-RNN approach.

Abstract

This paper presents our approach to the One-Minute Gradual-Emotion Recognition (OMG-Emotion) Challenge, focusing on dimensional emotion recognition through visual analysis of the provided emotion videos. The approach is based on a Convolutional and Recurrent (CNN-RNN) deep neural architecture we have developed for the relevant large AffWild Emotion Database. We extended and adapted this architecture, by letting a combination of multiple features generated in the CNN component be explored by RNN subnets. Our target has been to obtain best performance on the OMG-Emotion visual validation data set, while learning the respective visual training data set. Extended experimentation has led to best architectures for the estimation of the values of the valence and arousal emotion dimensions over these data sets.