LabellessFace: Fair Metric Learning for Face Recognition without Attribute Labels

TL;DR

LabellessFace introduces a novel fairness-aware metric learning framework for face recognition that reduces demographic bias without relying on demographic labels, using a new class favoritism metric and adaptive margin penalties.

Contribution

It proposes a label-free fairness enhancement method for face recognition, utilizing a new bias metric and adaptive margin penalties to improve fairness across all demographic groups.

Findings

Effective in reducing demographic bias without demographic labels

Maintains high authentication accuracy

Outperforms existing bias mitigation methods

Abstract

Demographic bias is one of the major challenges for face recognition systems. The majority of existing studies on demographic biases are heavily dependent on specific demographic groups or demographic classifier, making it difficult to address performance for unrecognised groups. This paper introduces ``LabellessFace'', a novel framework that improves demographic bias in face recognition without requiring demographic group labeling typically required for fairness considerations. We propose a novel fairness enhancement metric called the class favoritism level, which assesses the extent of favoritism towards specific classes across the dataset. Leveraging this metric, we introduce the fair class margin penalty, an extension of existing margin-based metric learning. This method dynamically adjusts learning parameters based on class favoritism levels, promoting fairness across all attributes. By treating each class as an individual in facial recognition systems, we facilitate learning that minimizes biases in authentication accuracy among individuals. Comprehensive experiments have demonstrated that our proposed method is effective for enhancing fairness while maintaining authentication accuracy.