Psychophysical Evaluation of Human Performance in Detecting Digital Face Image Manipulations

TL;DR

This study investigates human ability to detect digitally manipulated face images, revealing variability in detection performance and highlighting challenges in identifying certain manipulation types, which is crucial for security applications.

Contribution

It introduces a web-based psychophysical experiment to assess human proficiency in detecting various face image manipulations, filling a gap in existing research.

Findings

Detection performance varies widely among individuals.

Certain manipulation types are significantly harder to detect.

A potential metric for face manipulation detectability is discussed.

Abstract

In recent years, increasing deployment of face recognition technology in security-critical settings, such as border control or law enforcement, has led to considerable interest in the vulnerability of face recognition systems to attacks utilising legitimate documents, which are issued on the basis of digitally manipulated face images. As automated manipulation and attack detection remains a challenging task, conventional processes with human inspectors performing identity verification remain indispensable. These circumstances merit a closer investigation of human capabilities in detecting manipulated face images, as previous work in this field is sparse and often concentrated only on specific scenarios and biometric characteristics. This work introduces a web-based, remote visual discrimination experiment on the basis of principles adopted from the field of psychophysics and subsequently discusses interdisciplinary opportunities with the aim of examining human proficiency in detecting different types of digitally manipulated face images, specifically face swapping, morphing, and retouching. In addition to analysing appropriate performance measures, a possible metric of detectability is explored. Experimental data of 306 probands indicate that detection performance is widely distributed across the population and detection of certain types of face image manipulations is much more challenging than others.