Auditing and Mitigating Bias in Gender Classification Algorithms: A Data-Centric Approach

TL;DR

This paper audits gender classification datasets for bias, reveals significant underrepresentation, and introduces BalancedFace, a balanced dataset that substantially reduces bias in classifiers with minimal accuracy loss.

Contribution

It provides a comprehensive bias audit of existing datasets and creates BalancedFace, a new balanced dataset to mitigate bias in gender classification algorithms.

Findings

Bias exists in all audited datasets, especially for intersectional groups.

Training on BalancedFace reduces racial subgroup bias by over 50%.

BalancedFace improves fairness metrics with minimal impact on accuracy.

Abstract

Gender classification systems often inherit and amplify demographic imbalances in their training data. We first audit five widely used gender classification datasets, revealing that all suffer from significant intersectional underrepresentation. To measure the downstream impact of these flaws, we train identical MobileNetV2 classifiers on the two most balanced of these datasets, UTKFace and FairFace. Our fairness evaluation shows that even these models exhibit significant bias, misclassifying female faces at a higher rate than male faces and amplifying existing racial skew. To counter these data-induced biases, we construct BalancedFace, a new public dataset created by blending images from FairFace and UTKFace, supplemented with images from other collections to fill missing demographic gaps. It is engineered to equalize subgroup shares across 189 intersections of age, race, and gender using only real, unedited images. When a standard classifier is trained on BalancedFace, it reduces the maximum True Positive Rate gap across racial subgroups by over 50% and brings the average Disparate Impact score 63% closer to the ideal of 1.0 compared to the next-best dataset, all with a minimal loss of overall accuracy. These results underline the profound value of data-centric interventions and provide an openly available resource for fair gender classification research.