Interpretable Deep Neural Networks for Facial Expression and Dimensional Emotion Recognition in-the-wild

TL;DR

This paper introduces an interpretable deep neural network approach using a categorical GAN architecture for improved facial emotion recognition in-the-wild, leveraging dual annotations of Action Units and Valence Arousal.

Contribution

It proposes a novel dual-annotation dataset and a unique GAN-based architecture that jointly predicts and generates facial emotion representations, enhancing interpretability and performance.

Findings

GANs trained on separate annotations show promising results

Dual training improves emotion recognition accuracy

Generative capabilities assist in understanding model decisions

Abstract

In this project, we created a database with two types of annotations used in the emotion recognition domain : Action Units and Valence Arousal to try to achieve better results than with only one model. The originality of the approach is also based on the type of architecture used to perform the prediction of the emotions : a categorical Generative Adversarial Network. This kind of dual network can generate images based on the pictures from the new dataset thanks to its generative network and decide if an image is fake or real thanks to its discriminative network as well as help to predict the annotations for Action Units and Valence Arousal due to its categorical nature. GANs were trained on the Action Units model only, then the Valence Arousal model only and then on both the Action Units model and Valence Arousal model in order to test different parameters and understand their influence. The generative and discriminative aspects of the GANs have performed interesting results.