About Face: A Survey of Facial Recognition Evaluation

TL;DR

This survey reviews over 100 facial recognition datasets from 1976 to 2019, highlighting their historical context, biases, and the importance of transparent reporting to understand the technology's societal impact.

Contribution

It provides a comprehensive overview of facial recognition datasets, analyzing their historical development, biases, and the influence of societal factors on dataset construction.

Findings

Datasets are shaped by political and technological contexts.

Many datasets contain biases and problematic practices.

Transparency in dataset documentation is crucial for responsible use.

Abstract

We survey over 100 face datasets constructed between 1976 to 2019 of 145 million images of over 17 million subjects from a range of sources, demographics and conditions. Our historical survey reveals that these datasets are contextually informed, shaped by changes in political motivations, technological capability and current norms. We discuss how such influences mask specific practices (some of which may actually be harmful or otherwise problematic) and make a case for the explicit communication of such details in order to establish a more grounded understanding of the technology's function in the real world.