Hey Human, If your Facial Emotions are Uncertain, You Should Use Bayesian Neural Networks!

TL;DR

This paper demonstrates that Bayesian Neural Networks, using methods like MC-Dropout and Ensembles, effectively model uncertainty in facial emotion recognition, producing more human-aligned probability outputs and highlighting calibration challenges.

Contribution

It shows Bayesian Neural Networks can better handle uncertainty in facial emotion recognition, encouraging a shift from classical methods.

Findings

Bayesian methods produce probabilities closer to human expectations.

Calibration metrics behave unexpectedly due to multiple correct classes.

Bayesian Neural Networks effectively model aleatoric uncertainty.

Abstract

Facial emotion recognition is the task to classify human emotions in face images. It is a difficult task due to high aleatoric uncertainty and visual ambiguity. A large part of the literature aims to show progress by increasing accuracy on this task, but this ignores the inherent uncertainty and ambiguity in the task. In this paper we show that Bayesian Neural Networks, as approximated using MC-Dropout, MC-DropConnect, or an Ensemble, are able to model the aleatoric uncertainty in facial emotion recognition, and produce output probabilities that are closer to what a human expects. We also show that calibration metrics show strange behaviors for this task, due to the multiple classes that can be considered correct, which motivates future work. We believe our work will motivate other researchers to move away from Classical and into Bayesian Neural Networks.