Explaining Bias in Deep Face Recognition via Image Characteristics

TL;DR

This paper introduces a new framework to analyze how various image and demographic characteristics affect the fairness and performance of face recognition models across different groups.

Contribution

It provides a comprehensive analysis of how protected and non-protected attributes influence face recognition fairness, revealing complex interactions not seen in single-attribute studies.

Findings

Performance disparities vary with multi-attribute considerations.

Non-protected attributes significantly impact model performance.

Trends in single-attribute analysis may reverse in multi-attribute contexts.

Abstract

In this paper, we propose a novel explanatory framework aimed to provide a better understanding of how face recognition models perform as the underlying data characteristics (protected attributes: gender, ethnicity, age; non-protected attributes: facial hair, makeup, accessories, face orientation and occlusion, image distortion, emotions) on which they are tested change. With our framework, we evaluate ten state-of-the-art face recognition models, comparing their fairness in terms of security and usability on two data sets, involving six groups based on gender and ethnicity. We then analyze the impact of image characteristics on models performance. Our results show that trends appearing in a single-attribute analysis disappear or reverse when multi-attribute groups are considered, and that performance disparities are also related to non-protected attributes. Source code: https://cutt.ly/2XwRLiA.