Fix Representation (Optimally) Before Fairness: Finite-Sample Shrinkage Population Correction and the True Price of Fairness Under Subpopulation Shift

TL;DR

This paper demonstrates that subgroup proportion misrepresentation causes apparent fairness-accuracy tradeoffs and proposes a shrinkage correction method to accurately evaluate fairness interventions under subpopulation shift.

Contribution

It introduces a finite-sample shrinkage reweighting correction for subgroup proportions, clarifies the true impact of fairness constraints, and proposes an evaluation protocol fixing representation before fairness.

Findings

Shrinkage correction improves fairness evaluation accuracy.

Fairness interventions often appear beneficial due to baseline misweighting.

The proposed protocol reveals the true fairness-utility tradeoff.

Abstract

Machine learning practitioners frequently observe tension between predictive accuracy and group fairness constraints -- yet sometimes fairness interventions appear to improve accuracy. We show that both phenomena can be artifacts of training data that misrepresents subgroup proportions. Under subpopulation shift (stable within-group distributions, shifted group proportions), we establish: (i) full importance-weighted correction is asymptotically unbiased but finite-sample suboptimal; (ii) the optimal finite-sample correction is a shrinkage reweighting that interpolates between target and training mixtures; (iii) apparent "fairness helps accuracy" can arise from comparing fairness methods to an improperly-weighted baseline. We provide an actionable evaluation protocol: fix representation (optimally) before fairness -- compare fairness interventions against a shrinkage-corrected baseline to isolate the true, irreducible price of fairness. Experiments on synthetic and real-world benchmarks (Adult, COMPAS) validate our theoretical predictions and demonstrate that this protocol eliminates spurious tradeoffs, revealing the genuine fairness-utility frontier.