Alpha and Prejudice: Improving $\alpha$-sized Worst-case Fairness via Intrinsic Reweighting

TL;DR

This paper introduces a novel reweighting approach for $oldsymbol{ extit{ extalpha}}$-sized worst-case fairness that does not require demographic data, improving fairness in machine learning models through intrinsic importance and robust training schemes.

Contribution

It proposes a demographic-free reweighting method for $ extalpha$-sized worst-case fairness, including stochastic training and robustness to outliers, with theoretical and empirical validation.

Findings

Reweighting based on intrinsic importance improves fairness.

Stochastic learning simplifies training without performance loss.

Robust variant effectively handles outliers.

Abstract

Worst-case fairness with off-the-shelf demographics achieves group parity by maximizing the model utility of the worst-off group. Nevertheless, demographic information is often unavailable in practical scenarios, which impedes the use of such a direct max-min formulation. Recent advances have reframed this learning problem by introducing the lower bound of minimal partition ratio, denoted as $\alpha$, as side information, referred to as ``$\alpha$-sized worst-case fairness'' in this paper. We first justify the practical significance of this setting by presenting noteworthy evidence from the data privacy perspective, which has been overlooked by existing research. Without imposing specific requirements on loss functions, we propose reweighting the training samples based on their intrinsic importance to fairness. Given the global nature of the worst-case formulation, we further develop a stochastic learning scheme to simplify the training process without compromising model performance. Additionally, we address the issue of outliers and provide a robust variant to handle potential outliers during model training. Our theoretical analysis and experimental observations reveal the connections between the proposed approaches and existing ``fairness-through-reweighting'' studies, with extensive experimental results on fairness benchmarks demonstrating the superiority of our methods.