Towards Explaining Demographic Bias through the Eyes of Face Recognition Models

TL;DR

This paper introduces explainability tools that analyze how face recognition models behave differently across demographic groups by linking model responses to facial regions, helping to understand and address biases.

Contribution

It presents the first method to explain demographic biases in face recognition models by linking model behavior differences to specific facial regions using statistical analysis.

Findings

Models react differently to demographic groups in specific facial areas.

Analysis aligns with anthropometric and human judgment differences.

Provides insights for reducing bias in face recognition systems.

Abstract

Biases inherent in both data and algorithms make the fairness of widespread machine learning (ML)-based decision-making systems less than optimal. To improve the trustfulness of such ML decision systems, it is crucial to be aware of the inherent biases in these solutions and to make them more transparent to the public and developers. In this work, we aim at providing a set of explainability tool that analyse the difference in the face recognition models' behaviors when processing different demographic groups. We do that by leveraging higher-order statistical information based on activation maps to build explainability tools that link the FR models' behavior differences to certain facial regions. The experimental results on two datasets and two face recognition models pointed out certain areas of the face where the FR models react differently for certain demographic groups compared to reference groups. The outcome of these analyses interestingly aligns well with the results of studies that analyzed the anthropometric differences and the human judgment differences on the faces of different demographic groups. This is thus the first study that specifically tries to explain the biased behavior of FR models on different demographic groups and link it directly to the spatial facial features. The code is publicly available here.