Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition

TL;DR

This paper examines how occlusions in facial images influence demographic bias in face recognition systems, revealing that occlusions worsen biases and affect different groups unequally, especially impacting African individuals.

Contribution

It introduces a new metric, FOIR, to quantify occlusion impact and demonstrates that occlusions increase demographic biases in face recognition models.

Findings

Occlusions increase bias metrics like FMR, FNMR, and accuracy dispersion.

Models assign higher importance to occlusions for African individuals.

Occlusions exacerbate existing demographic biases in face recognition.

Abstract

This study investigates the effects of occlusions on the fairness of face recognition systems, particularly focusing on demographic biases. Using the Racial Faces in the Wild (RFW) dataset and synthetically added realistic occlusions, we evaluate their effect on the performance of face recognition models trained on the BUPT-Balanced and BUPT-GlobalFace datasets. We note increases in the dispersion of FMR, FNMR, and accuracy alongside decreases in fairness according to Equilized Odds, Demographic Parity, STD of Accuracy, and Fairness Discrepancy Rate. Additionally, we utilize a pixel attribution method to understand the importance of occlusions in model predictions, proposing a new metric, Face Occlusion Impact Ratio (FOIR), that quantifies the extent to which occlusions affect model performance across different demographic groups. Our results indicate that occlusions exacerbate existing demographic biases, with models placing higher importance on occlusions in an unequal fashion, particularly affecting African individuals more severely.