Fairness in Biometrics: a figure of merit to assess biometric verification systems

TL;DR

This paper introduces the Fairness Discrepancy Rate (FDR), a novel metric for evaluating and comparing fairness in biometric verification systems, demonstrated through synthetic and face biometric use cases involving gender and race demographics.

Contribution

The paper presents the first figure of merit, FDR, to assess and compare fairness across biometric verification systems, addressing a critical social impact aspect.

Findings

FDR effectively distinguishes fair and unfair biometric systems.

Synthetic use case demonstrates FDR's ability to detect extreme fairness behaviors.

Face biometric evaluation shows FDR's utility in real-world demographic scenarios.

Abstract

Machine learning-based (ML) systems are being largely deployed since the last decade in a myriad of scenarios impacting several instances in our daily lives. With this vast sort of applications, aspects of fairness start to rise in the spotlight due to the social impact that this can get in minorities. In this work aspects of fairness in biometrics are addressed. First, we introduce the first figure of merit that is able to evaluate and compare fairness aspects between multiple biometric verification systems, the so-called Fairness Discrepancy Rate (FDR). A use case with two synthetic biometric systems is introduced and demonstrates the potential of this figure of merit in extreme cases of fair and unfair behavior. Second, a use case using face biometrics is presented where several systems are evaluated compared with this new figure of merit using three public datasets exploring gender and race demographics.