FaceX: Understanding Face Attribute Classifiers through Summary Model Explanations

TL;DR

FaceX introduces a novel summary explanation method for face attribute classifiers, enabling comprehensive, region-based interpretability and bias detection across large facial datasets, surpassing traditional pixel attribution approaches.

Contribution

This paper presents FaceX, the first method to provide summary model explanations for face attribute classifiers, improving interpretability and bias detection at a regional level.

Findings

Effective identification of model biases across multiple benchmarks

High interpretability through region-level activation visualization

Robust performance in bias mitigation scenarios

Abstract

EXplainable Artificial Intelligence (XAI) approaches are widely applied for identifying fairness issues in Artificial Intelligence (AI) systems. However, in the context of facial analysis, existing XAI approaches, such as pixel attribution methods, offer explanations for individual images, posing challenges in assessing the overall behavior of a model, which would require labor-intensive manual inspection of a very large number of instances and leaving to the human the task of drawing a general impression of the model behavior from the individual outputs. Addressing this limitation, we introduce FaceX, the first method that provides a comprehensive understanding of face attribute classifiers through summary model explanations. Specifically, FaceX leverages the presence of distinct regions across all facial images to compute a region-level aggregation of model activations, allowing for the visualization of the model's region attribution across 19 predefined regions of interest in facial images, such as hair, ears, or skin. Beyond spatial explanations, FaceX enhances interpretability by visualizing specific image patches with the highest impact on the model's decisions for each facial region within a test benchmark. Through extensive evaluation in various experimental setups, including scenarios with or without intentional biases and mitigation efforts on four benchmarks, namely CelebA, FairFace, CelebAMask-HQ, and Racial Faces in the Wild, FaceX demonstrates high effectiveness in identifying the models' biases.