The More Secure, The Less Equally Usable: Gender and Ethnicity (Un)fairness of Deep Face Recognition along Security Thresholds

TL;DR

This paper investigates how increasing security thresholds in face recognition systems amplifies demographic disparities, revealing fairness issues especially in high-security scenarios.

Contribution

It provides a comprehensive analysis of demographic disparities across multiple face recognition models and security thresholds, highlighting fairness concerns in high-stakes applications.

Findings

Higher security levels increase demographic disparities in face recognition accuracy.

Disparities are consistent across different models and demographic groups.

Fairness issues are more pronounced in high-security settings.

Abstract

Face biometrics are playing a key role in making modern smart city applications more secure and usable. Commonly, the recognition threshold of a face recognition system is adjusted based on the degree of security for the considered use case. The likelihood of a match can be for instance decreased by setting a high threshold in case of a payment transaction verification. Prior work in face recognition has unfortunately showed that error rates are usually higher for certain demographic groups. These disparities have hence brought into question the fairness of systems empowered with face biometrics. In this paper, we investigate the extent to which disparities among demographic groups change under different security levels. Our analysis includes ten face recognition models, three security thresholds, and six demographic groups based on gender and ethnicity. Experiments show that the higher the security of the system is, the higher the disparities in usability among demographic groups are. Compelling unfairness issues hence exist and urge countermeasures in real-world high-stakes environments requiring severe security levels.