An Explainable Fast Deep Neural Network for Emotion Recognition

TL;DR

This paper introduces an explainable AI approach for emotion recognition from video, optimizing facial landmarks to improve classifier accuracy and interpretability while reducing noise and computational costs.

Contribution

It presents a novel explainability algorithm that identifies key facial landmarks for emotion classification, enhancing model performance and interpretability.

Findings

Optimized facial landmarks improve emotion recognition accuracy.

Explainability reduces noise impact and computational cost.

Facial landmarks relevance varies across emotions.

Abstract

In the context of artificial intelligence, the inherent human attribute of engaging in logical reasoning to facilitate decision-making is mirrored by the concept of explainability, which pertains to the ability of a model to provide a clear and interpretable account of how it arrived at a particular outcome. This study explores explainability techniques for binary deep neural architectures in the framework of emotion classification through video analysis. We investigate the optimization of input features to binary classifiers for emotion recognition, with face landmarks detection using an improved version of the Integrated Gradients explainability method. The main contribution of this paper consists in the employment of an innovative explainable artificial intelligence algorithm to understand the crucial facial landmarks movements during emotional feeling, using this information also for improving the performances of deep learning-based emotion classifiers. By means of explainability, we can optimize the number and the position of the facial landmarks used as input features for facial emotion recognition, lowering the impact of noisy landmarks and thus increasing the accuracy of the developed models. In order to test the effectiveness of the proposed approach, we considered a set of deep binary models for emotion classification trained initially with a complete set of facial landmarks, which are progressively reduced based on a suitable optimization procedure. The obtained results prove the robustness of the proposed explainable approach in terms of understanding the relevance of the different facial points for the different emotions, also improving the classification accuracy and diminishing the computational cost.