Balancing the Scales: Enhancing Fairness in Facial Expression Recognition with Latent Alignment

TL;DR

This paper proposes a novel latent space representation learning approach to mitigate demographic biases in facial expression recognition, aiming to improve fairness and accuracy across diverse socio-cultural groups.

Contribution

It introduces a latent alignment method that reduces demographic bias in FER models, addressing dataset imbalance and annotation biases.

Findings

Improved fairness across demographic groups

Enhanced recognition accuracy in diverse populations

Reduced bias in facial expression datasets

Abstract

Automatically recognizing emotional intent using facial expression has been a thoroughly investigated topic in the realm of computer vision. Facial Expression Recognition (FER), being a supervised learning task, relies heavily on substantially large data exemplifying various socio-cultural demographic attributes. Over the past decade, several real-world in-the-wild FER datasets that have been proposed were collected through crowd-sourcing or web-scraping. However, most of these practically used datasets employ a manual annotation methodology for labeling emotional intent, which inherently propagates individual demographic biases. Moreover, these datasets also lack an equitable representation of various socio-cultural demographic groups, thereby inducing a class imbalance. Bias analysis and its mitigation have been investigated across multiple domains and problem settings, however, in the FER domain, this is a relatively lesser explored area. This work leverages representation learning based on latent spaces to mitigate bias in facial expression recognition systems, thereby enhancing a deep learning model's fairness and overall accuracy.