Bridging the gap in FER: addressing age bias in deep learning

TL;DR

This paper investigates age bias in deep learning-based facial expression recognition systems, identifies disparities using explainable AI, and proposes bias mitigation strategies that improve accuracy for elderly individuals.

Contribution

It provides a comprehensive analysis of age bias in FER, introduces three mitigation methods, and demonstrates their effectiveness using large-scale datasets.

Findings

Biases are systematic for elderly individuals, especially for neutral, sadness, and anger expressions.

Age-aware training strategies improve recognition accuracy for underrepresented age groups.

Models trained with age information attend to more relevant facial regions across ages.

Abstract

Facial Expression Recognition (FER) systems based on deep learning have achieved impressive performance in recent years. However, these models often exhibit demographic biases, particularly with respect to age, which can compromise their fairness and reliability. In this work, we present a comprehensive study of age-related bias in deep FER models, with a particular focus on the elderly population. We first investigate whether recognition performance varies across age groups, which expressions are most affected, and whether model attention differs depending on age. Using Explainable AI (XAI) techniques, we identify systematic disparities in expression recognition and attention patterns, especially for "neutral", "sadness", and "anger" in elderly individuals. Based on these findings, we propose and evaluate three bias mitigation strategies: Multi-task Learning, Multi-modal Input, and Age-weighted Loss. Our models are trained on a large-scale dataset, AffectNet, with automatically estimated age labels and validated on balanced benchmark datasets that include underrepresented age groups. Results show consistent improvements in recognition accuracy for elderly individuals, particularly for the most error-prone expressions. Saliency heatmap analysis reveals that models trained with age-aware strategies attend to more relevant facial regions for each age group, helping to explain the observed improvements. These findings suggest that age-related bias in FER can be effectively mitigated using simple training modifications, and that even approximate demographic labels can be valuable for promoting fairness in large-scale affective computing systems.