Beard Segmentation and Recognition Bias

TL;DR

This paper investigates how facial hair affects face recognition accuracy, introduces a segmentation model for facial hair, and proposes an adaptive thresholding method to reduce bias across different facial hairstyle categories.

Contribution

It presents a novel facial hair segmentation model and a bias mitigation scheme for face recognition systems based on facial hairstyle similarity.

Findings

Facial hairstyle significantly impacts false match rates.

Adaptive thresholding reduces recognition bias across hairstyle categories.

Facial hair segmentation model improves analysis of hairstyle influence.

Abstract

A person's facial hairstyle, such as presence and size of beard, can significantly impact face recognition accuracy. There are publicly-available deep networks that achieve reasonable accuracy at binary attribute classification, such as beard / no beard, but few if any that segment the facial hair region. To investigate the effect of facial hair in a rigorous manner, we first created a set of fine-grained facial hair annotations to train a segmentation model and evaluate its accuracy across African-American and Caucasian face images. We then use our facial hair segmentations to categorize image pairs according to the degree of difference or similarity in the facial hairstyle. We find that the False Match Rate (FMR) for image pairs with different categories of facial hairstyle varies by a factor of over 10 for African-American males and over 25 for Caucasian males. To reduce the bias across image pairs with different facial hairstyles, we propose a scheme for adaptive thresholding based on facial hairstyle similarity. Evaluation on a subject-disjoint set of images shows that adaptive similarity thresholding based on facial hairstyles of the image pair reduces the ratio between the highest and lowest FMR across facial hairstyle categories for African-American from 10.7 to 1.8 and for Caucasians from 25.9 to 1.3. Facial hair annotations and facial hair segmentation model will be publicly available.