Invariant Feature Regularization for Fair Face Recognition

TL;DR

This paper introduces Invariant Feature Regularization (INV-REG), an unsupervised method to mitigate demographic bias in face recognition by generating self-annotated confounders, leading to improved fairness across demographic groups.

Contribution

The paper proposes a novel unsupervised approach to deconfound demographic biases in face recognition, enhancing fairness without requiring costly demographic annotations.

Findings

INV-REG improves face recognition fairness across demographic groups.

Combining INV-REG with existing methods achieves state-of-the-art results.

The method is orthogonal and complementary to current face recognition baselines.

Abstract

Fair face recognition is all about learning invariant feature that generalizes to unseen faces in any demographic group. Unfortunately, face datasets inevitably capture the imbalanced demographic attributes that are ubiquitous in real-world observations, and the model learns biased feature that generalizes poorly in the minority group. We point out that the bias arises due to the confounding demographic attributes, which mislead the model to capture the spurious demographic-specific feature. The confounding effect can only be removed by causal intervention, which requires the confounder annotations. However, such annotations can be prohibitively expensive due to the diversity of the demographic attributes. To tackle this, we propose to generate diverse data partitions iteratively in an unsupervised fashion. Each data partition acts as a self-annotated confounder, enabling our Invariant Feature Regularization (INV-REG) to deconfound. INV-REG is orthogonal to existing methods, and combining INV-REG with two strong baselines (Arcface and CIFP) leads to new state-of-the-art that improves face recognition on a variety of demographic groups. Code is available at https://github.com/PanasonicConnect/InvReg.