Evaluation of Human and Machine Face Detection using a Novel Distinctive Human Appearance Dataset

TL;DR

This paper introduces a new dataset capturing diverse human appearances to evaluate face detection systems, revealing current models' limitations in handling appearance diversity and emphasizing the need for more inclusive algorithms.

Contribution

The creation of the Distinctive Human Appearance dataset and its use to assess and highlight the shortcomings of existing face detection models in diverse real-world scenarios.

Findings

Face detection models perform poorly on diverse appearances.

Current datasets lack representation of low-frequency human features.

Evaluation highlights the need for more inclusive face detection systems.

Abstract

Face detection is a long-standing challenge in the field of computer vision, with the ultimate goal being to accurately localize human faces in an unconstrained environment. There are significant technical hurdles in making these systems accurate due to confounding factors related to pose, image resolution, illumination, occlusion, and viewpoint [44]. That being said, with recent developments in machine learning, face-detection systems have achieved extraordinary accuracy, largely built on data-driven deep-learning models [70]. Though encouraging, a critical aspect that limits face-detection performance and social responsibility of deployed systems is the inherent diversity of human appearance. Every human appearance reflects something unique about a person, including their heritage, identity, experiences, and visible manifestations of self-expression. However, there are questions about how well face-detection systems perform when faced with varying face size and shape, skin color, body modification, and body ornamentation. Towards this goal, we collected the Distinctive Human Appearance dataset, an image set that represents appearances with low frequency and that tend to be undersampled in face datasets. Then, we evaluated current state-of-the-art face-detection models in their ability to detect faces in these images. The evaluation results show that face-detection algorithms do not generalize well to these diverse appearances. Evaluating and characterizing the state of current face-detection models will accelerate research and development towards creating fairer and more accurate face-detection systems.