Fair Contrastive Learning for Facial Attribute Classification

TL;DR

This paper introduces a fair supervised contrastive learning method that reduces bias related to sensitive attributes in facial attribute classification, improving fairness without sacrificing accuracy.

Contribution

It proposes a novel Fair Supervised Contrastive Loss (FSCL) that promotes fair representations by penalizing sensitive attribute information and introduces group-wise normalization to address demographic disparities.

Findings

Outperforms state-of-the-art methods in fairness and accuracy trade-offs

Robust to data bias intensity and incomplete supervision

Effective in reducing demographic disparities in facial attribute classification

Abstract

Learning visual representation of high quality is essential for image classification. Recently, a series of contrastive representation learning methods have achieved preeminent success. Particularly, SupCon outperformed the dominant methods based on cross-entropy loss in representation learning. However, we notice that there could be potential ethical risks in supervised contrastive learning. In this paper, we for the first time analyze unfairness caused by supervised contrastive learning and propose a new Fair Supervised Contrastive Loss (FSCL) for fair visual representation learning. Inheriting the philosophy of supervised contrastive learning, it encourages representation of the same class to be closer to each other than that of different classes, while ensuring fairness by penalizing the inclusion of sensitive attribute information in representation. In addition, we introduce a group-wise normalization to diminish the disparities of intra-group compactness and inter-class separability between demographic groups that arouse unfair classification. Through extensive experiments on CelebA and UTK Face, we validate that the proposed method significantly outperforms SupCon and existing state-of-the-art methods in terms of the trade-off between top-1 accuracy and fairness. Moreover, our method is robust to the intensity of data bias and effectively works in incomplete supervised settings. Our code is available at https://github.com/sungho-CoolG/FSCL.