Impact of Blur and Resolution on Demographic Disparities in 1-to-Many Facial Identification

TL;DR

This study investigates how blur and low resolution in surveillance images affect demographic disparities in 1-to-many facial identification accuracy, revealing increased false positives especially for certain groups and highlighting differences from 1-to-1 matching.

Contribution

It introduces new metrics for analyzing 1-to-many face recognition accuracy across demographics under degraded image conditions and demonstrates the impact of image quality on false positive rates.

Findings

Blur and low resolution significantly increase false positive rates.

Demographic disparities are larger under degraded image conditions.

1-to-many accuracy can deteriorate more than 1-to-1 accuracy with image quality loss.

Abstract

Most studies to date that have examined demographic variations in face recognition accuracy have analyzed 1-to-1 matching accuracy, using images that could be described as "government ID quality". This paper analyzes the accuracy of 1-to-many facial identification across demographic groups, and in the presence of blur and reduced resolution in the probe image as might occur in "surveillance camera quality" images. Cumulative match characteristic curves (CMC) are not appropriate for comparing propensity for rank-one recognition errors across demographics, and so we use three metrics for our analysis: (1) the well-known d' metric between mated and non-mated score distributions, and introduced in this work, (2) absolute score difference between thresholds in the high-similarity tail of the non-mated and the low-similarity tail of the mated distribution, and (3) distribution of (mated - non-mated rank-one scores) across the set of probe images. We find that demographic variation in 1-to-many accuracy does not entirely follow what has been observed in 1-to-1 matching accuracy. Also, different from 1-to-1 accuracy, demographic comparison of 1-to-many accuracy can be affected by different numbers of identities and images across demographics. More importantly, we show that increased blur in the probe image, or reduced resolution of the face in the probe image, can significantly increase the false positive identification rate. And we show that the demographic variation in these high blur or low resolution conditions is much larger for male / female than for African-American / Caucasian. The point that 1-to-many accuracy can potentially collapse in the context of processing "surveillance camera quality" probe images against a "government ID quality" gallery is an important one.