An Experimental Evaluation of Covariates Effects on Unconstrained Face Verification

TL;DR

This study evaluates how various covariates impact deep neural network performance in unconstrained face verification and explores using covariate information, like gender, to enhance verification accuracy on large datasets.

Contribution

The paper provides a comprehensive analysis of covariate effects on face verification and demonstrates how gender information can improve dataset quality and model performance.

Findings

Covariates significantly affect face verification performance.

Gender information can improve dataset quality and model accuracy.

Performance gains are observed at very low false acceptance rates.

Abstract

Covariates are factors that have a debilitating influence on face verification performance. In this paper, we comprehensively study two covariate related problems for unconstrained face verification: first, how covariates affect the performance of deep neural networks on the large-scale unconstrained face verification problem; second, how to utilize covariates to improve verification performance. To study the first problem, we implement five state-of-the-art deep convolutional networks (DCNNs) for face verification and evaluate them on three challenging covariates datasets. In total, seven covariates are considered: pose (yaw and roll), age, facial hair, gender, indoor/outdoor, occlusion (nose and mouth visibility, eyes visibility, and forehead visibility), and skin tone. These covariates cover both intrinsic subject-specific characteristics and extrinsic factors of faces. Some of the results confirm and extend the findings of previous studies, others are new findings that were rarely mentioned previously or did not show consistent trends. For the second problem, we demonstrate that with the assistance of gender information, the quality of a pre-curated noisy large-scale face dataset for face recognition can be further improved. After retraining the face recognition model using the curated data, performance improvement is observed at low False Acceptance Rates (FARs) (FAR=$10^{-5}$, $10^{-6}$, $10^{-7}$).