Gendered Differences in Face Recognition Accuracy Explained by Hairstyles, Makeup, and Facial Morphology

TL;DR

This study investigates the causes of lower face recognition accuracy for females, finding that hairstyle, makeup, and facial morphology significantly influence bias, and controlling these factors improves accuracy for women.

Contribution

It is the first experimental analysis to identify specific factors like hairstyle, makeup, and facial morphology that contribute to gender bias in face recognition accuracy.

Findings

Controlling for visible face area reduces female false non-match rates.

Balanced makeup datasets improve female recognition accuracy.

Female facial images are inherently more similar, affecting false match rates.

Abstract

Media reports have accused face recognition of being ''biased'', ''sexist'' and ''racist''. There is consensus in the research literature that face recognition accuracy is lower for females, who often have both a higher false match rate and a higher false non-match rate. However, there is little published research aimed at identifying the cause of lower accuracy for females. For instance, the 2019 Face Recognition Vendor Test that documents lower female accuracy across a broad range of algorithms and datasets also lists ''Analyze cause and effect'' under the heading ''What we did not do''. We present the first experimental analysis to identify major causes of lower face recognition accuracy for females on datasets where previous research has observed this result. Controlling for equal amount of visible face in the test images mitigates the apparent higher false non-match rate for females. Additional analysis shows that makeup-balanced datasets further improves females to achieve lower false non-match rates. Finally, a clustering experiment suggests that images of two different females are inherently more similar than of two different males, potentially accounting for a difference in false match rates.