Fairness without Sensitive Attributes via Knowledge Sharing

TL;DR

This paper introduces Reckoner, a hierarchical classifier that promotes fairness without relying on sensitive attributes by sharing knowledge between high-confidence and low-confidence data models, improving fairness and accuracy.

Contribution

The paper proposes a novel confidence-based hierarchical classifier, Reckoner, that enhances fairness without sensitive attributes by leveraging knowledge sharing between data subsets.

Findings

Reckoner outperforms state-of-the-art methods in fairness and accuracy.

Bias increases in high-confidence subsets with biased labels.

Knowledge sharing reduces bias gap across demographic groups.

Abstract

While model fairness improvement has been explored previously, existing methods invariably rely on adjusting explicit sensitive attribute values in order to improve model fairness in downstream tasks. However, we observe a trend in which sensitive demographic information becomes inaccessible as public concerns around data privacy grow. In this paper, we propose a confidence-based hierarchical classifier structure called "Reckoner" for reliable fair model learning under the assumption of missing sensitive attributes. We first present results showing that if the dataset contains biased labels or other hidden biases, classifiers significantly increase the bias gap across different demographic groups in the subset with higher prediction confidence. Inspired by these findings, we devised a dual-model system in which a version of the model initialised with a high-confidence data subset learns from a version of the model initialised with a low-confidence data subset, enabling it to avoid biased predictions. Our experimental results show that Reckoner consistently outperforms state-of-the-art baselines in COMPAS dataset and New Adult dataset, considering both accuracy and fairness metrics.