Lights, Camera, Matching: The Role of Image Illumination in Fair Face Recognition

TL;DR

This paper investigates how facial illumination affects face recognition fairness across demographic groups and proposes methods to balance brightness, reducing accuracy disparities between Caucasian and African American females.

Contribution

It introduces brightness balancing techniques based on median and distribution of pixel values to improve fairness in face recognition systems.

Findings

Balancing brightness reduces demographic accuracy gaps by up to 57.6%.

Similarity scores improve for both demographic groups after brightness balancing.

Brightness balancing enhances face recognition fairness across demographic groups.

Abstract

Facial brightness is a key image quality factor impacting face recognition accuracy differentials across demographic groups. In this work, we aim to decrease the accuracy gap between the similarity score distributions for Caucasian and African American female mated image pairs, as measured by d' between distributions. To balance brightness across demographic groups, we conduct three experiments, interpreting brightness in the face skin region either as median pixel value or as the distribution of pixel values. Balancing based on median brightness alone yields up to a 46.8% decrease in d', while balancing based on brightness distribution yields up to a 57.6% decrease. In all three cases, the similarity scores of the individual distributions improve, with mean scores maximally improving 5.9% for Caucasian females and 3.7% for African American females.