Bias Analysis for Synthetic Face Detection: A Case Study of the Impact of Facial Attributes

TL;DR

This paper introduces a framework for analyzing bias in synthetic face detection models, revealing that these detectors are often biased towards specific facial attributes and exploring the origins of such biases.

Contribution

The work presents an evaluation framework using synthetic data with balanced attributes to analyze bias in face detectors, along with an extensive case study of five state-of-the-art models.

Findings

Detectors are biased towards certain facial attributes.

Bias correlates with training data attribute distributions.

Analysis of activation maps reveals bias origins.

Abstract

Bias analysis for synthetic face detection is bound to become a critical topic in the coming years. Although many detection models have been developed and several datasets have been released to reliably identify synthetic content, one crucial aspect has been largely overlooked: these models and training datasets can be biased, leading to failures in detection for certain demographic groups and raising significant social, legal, and ethical issues. In this work, we introduce an evaluation framework to contribute to the analysis of bias of synthetic face detectors with respect to several facial attributes. This framework exploits synthetic data generation, with evenly distributed attribute labels, for mitigating any skew in the data that could otherwise influence the outcomes of bias analysis. We build on the proposed framework to provide an extensive case study of the bias level of five state-of-the-art detectors in synthetic datasets with 25 controlled facial attributes. While the results confirm that, in general, synthetic face detectors are biased towards the presence/absence of specific facial attributes, our study also sheds light on the origins of the observed bias through the analysis of the correlations with the balancing of facial attributes in the training sets of the detectors, and the analysis of detectors activation maps in image pairs with controlled attribute modifications.