Harnessing Unrecognizable Faces for Improving Face Recognition

TL;DR

This paper introduces a recognizability measure for face images based on deep neural network embeddings, improving face recognition accuracy by accounting for unrecognizable faces across various challenging conditions.

Contribution

It proposes a novel recognizability metric derived from embedding space clustering, enhancing face recognition systems by reducing errors caused by unrecognizable faces.

Findings

58% error reduction at FAR=1e-5 in single-image recognition

24% error reduction at FAR=1e-5 in set-based recognition

Effective handling of unrecognizable faces under various conditions

Abstract

The common implementation of face recognition systems as a cascade of a detection stage and a recognition or verification stage can cause problems beyond failures of the detector. When the detector succeeds, it can detect faces that cannot be recognized, no matter how capable the recognition system. Recognizability, a latent variable, should therefore be factored into the design and implementation of face recognition systems. We propose a measure of recognizability of a face image that leverages a key empirical observation: an embedding of face images, implemented by a deep neural network trained using mostly recognizable identities, induces a partition of the hypersphere whereby unrecognizable identities cluster together. This occurs regardless of the phenomenon that causes a face to be unrecognizable, it be optical or motion blur, partial occlusion, spatial quantization, poor illumination. Therefore, we use the distance from such an "unrecognizable identity" as a measure of recognizability, and incorporate it in the design of the over-all system. We show that accounting for recognizability reduces error rate of single-image face recognition by 58% at FAR=1e-5 on the IJB-C Covariate Verification benchmark, and reduces verification error rate by 24% at FAR=1e-5 in set-based recognition on the IJB-C benchmark.