An Evaluation of Forensic Facial Recognition

TL;DR

This paper evaluates the performance of facial recognition systems in challenging forensic scenarios using synthetic and real datasets, revealing significant accuracy drops compared to controlled conditions.

Contribution

It introduces a large-scale synthetic facial dataset and a controlled forensic lineup for realistic evaluation of recognition systems.

Findings

Recognition accuracy drops from over 95% to around 65% in forensic scenarios.

Synthetic dataset enables controlled testing of recognition under real-world conditions.

Neural-based recognition systems perform significantly worse in forensic-like conditions.

Abstract

Recent advances in machine learning and computer vision have led to reported facial recognition accuracies surpassing human performance. We question if these systems will translate to real-world forensic scenarios in which a potentially low-resolution, low-quality, partially-occluded image is compared against a standard facial database. We describe the construction of a large-scale synthetic facial dataset along with a controlled facial forensic lineup, the combination of which allows for a controlled evaluation of facial recognition under a range of real-world conditions. Using this synthetic dataset, and a popular dataset of real faces, we evaluate the accuracy of two popular neural-based recognition systems. We find that previously reported face recognition accuracies of more than 95% drop to as low as 65% in this more challenging forensic scenario.