PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognition

TL;DR

PASS is a descriptor-based adversarial method that reduces sensitive attribute encoding in face recognition descriptors, mitigating bias without requiring end-to-end retraining, and maintains high verification accuracy.

Contribution

It introduces PASS, a novel approach that suppresses protected attribute information in face descriptors, enabling bias mitigation on pre-trained networks without full retraining.

Findings

Reduces gender and skintone bias in face descriptors

Outperforms baselines in bias reduction on IJB-C dataset

Maintains high verification accuracy

Abstract

Face recognition networks encode information about sensitive attributes while being trained for identity classification. Such encoding has two major issues: (a) it makes the face representations susceptible to privacy leakage (b) it appears to contribute to bias in face recognition. However, existing bias mitigation approaches generally require end-to-end training and are unable to achieve high verification accuracy. Therefore, we present a descriptor-based adversarial de-biasing approach called `Protected Attribute Suppression System (PASS)'. PASS can be trained on top of descriptors obtained from any previously trained high-performing network to classify identities and simultaneously reduce encoding of sensitive attributes. This eliminates the need for end-to-end training. As a component of PASS, we present a novel discriminator training strategy that discourages a network from encoding protected attribute information. We show the efficacy of PASS to reduce gender and skintone information in descriptors from SOTA face recognition networks like Arcface. As a result, PASS descriptors outperform existing baselines in reducing gender and skintone bias on the IJB-C dataset, while maintaining a high verification accuracy.