A Feature-level Bias Evaluation Framework for Facial Expression Recognition Models

TL;DR

This paper introduces a feature-level bias evaluation framework for facial expression recognition models that does not require demographic labels, enabling more accurate bias assessment and statistical validation across multiple attributes and architectures.

Contribution

The authors propose a novel bias evaluation framework that operates without demographic labels and includes a statistical significance module, advancing fairness analysis in FER models.

Findings

The framework more effectively evaluates demographic biases than existing pseudo-label methods.

Many FER studies lack statistical testing, risking unreliable bias reports.

Biases are prominent across various attributes and architectures, highlighting fairness concerns.

Abstract

Recent studies on fairness have shown that Facial Expression Recognition (FER) models exhibit biases toward certain visually perceived demographic groups. However, the limited availability of human-annotated demographic labels in public FER datasets has constrained the scope of such bias analysis. To overcome this limitation, some prior works have resorted to pseudo-demographic labels, which may distort bias evaluation results. Alternatively, in this paper, we propose a feature-level bias evaluation framework for evaluating demographic biases in FER models under the setting where demographic labels are unavailable in the test set. Extensive experiments demonstrate that our method more effectively evaluates demographic biases compared to existing approaches that rely on pseudo-demographic labels. Furthermore, we observe that many existing studies do not include statistical testing in their bias evaluations, raising concerns that some reported biases may not be statistically significant but rather due to randomness. To address this issue, we introduce a plug-and-play statistical module to ensure the statistical significance of biased evaluation results. A comprehensive bias analysis based on the proposed module is then conducted across three sensitive attributes (age, gender, and race), seven facial expressions, and multiple network architectures on a large-scale dataset, revealing the prominent demographic biases in FER and providing insights on selecting a fairer network architecture.