Towards Inclusive Face Recognition Through Synthetic Ethnicity Alteration

TL;DR

This paper investigates synthetic ethnicity alteration in face images using GANs to create diverse datasets, aiming to reduce bias in face recognition systems by analyzing the quality and realism of altered images.

Contribution

It introduces a methodology for synthetic ethnicity and skin tone modification using GANs, and evaluates their impact on face recognition performance and dataset diversity.

Findings

GAN-based ethnicity alteration can produce realistic face images.

Diverse datasets improve face recognition fairness.

Analysis of skin tone representation using ITA enhances dataset quality.

Abstract

Numerous studies have shown that existing Face Recognition Systems (FRS), including commercial ones, often exhibit biases toward certain ethnicities due to under-represented data. In this work, we explore ethnicity alteration and skin tone modification using synthetic face image generation methods to increase the diversity of datasets. We conduct a detailed analysis by first constructing a balanced face image dataset representing three ethnicities: Asian, Black, and Indian. We then make use of existing Generative Adversarial Network-based (GAN) image-to-image translation and manifold learning models to alter the ethnicity from one to another. A systematic analysis is further conducted to assess the suitability of such datasets for FRS by studying the realistic skin-tone representation using Individual Typology Angle (ITA). Further, we also analyze the quality characteristics using existing Face image quality assessment (FIQA) approaches. We then provide a holistic FRS performance analysis using four different systems. Our findings pave the way for future research works in (i) developing both specific ethnicity and general (any to any) ethnicity alteration models, (ii) expanding such approaches to create databases with diverse skin tones, (iii) creating datasets representing various ethnicities which further can help in mitigating bias while addressing privacy concerns.