Classifying and Visualizing Emotions with Emotional DAN

TL;DR

This paper introduces EmotionalDAN, a novel emotion classification model that incorporates facial landmarks into the loss function, outperforming existing methods and providing interpretable visualizations of facial features related to emotions.

Contribution

We extend the Deep Alignment Network (DAN) with facial landmark-based loss, creating EmotionalDAN, which improves emotion recognition accuracy and interpretability.

Findings

Outperforms state-of-the-art on benchmark datasets by up to 5%.

Effectively identifies facial landmarks associated with emotions.

Provides visual explanations of emotion classification decisions.

Abstract

Classification of human emotions remains an important and challenging task for many computer vision algorithms, especially in the era of humanoid robots which coexist with humans in their everyday life. Currently proposed methods for emotion recognition solve this task using multi-layered convolutional networks that do not explicitly infer any facial features in the classification phase. In this work, we postulate a fundamentally different approach to solve emotion recognition task that relies on incorporating facial landmarks as a part of the classification loss function. To that end, we extend a recently proposed Deep Alignment Network (DAN) with a term related to facial features. Thanks to this simple modification, our model called EmotionalDAN is able to outperform state-of-the-art emotion classification methods on two challenging benchmark dataset by up to 5%. Furthermore, we visualize image regions analyzed by the network when making a decision and the results indicate that our EmotionalDAN model is able to correctly identify facial landmarks responsible for expressing the emotions.