The Statistical Fairness-Accuracy Frontier

TL;DR

This paper investigates the fairness-accuracy tradeoff in predictive models serving multiple groups, providing finite-sample bounds, optimal estimators, and confidence bands to guide fair decision-making.

Contribution

It introduces finite-sample analysis of the fairness-accuracy frontier, deriving optimal estimators and sample strategies to approximate the tradeoff in limited data settings.

Findings

Derived worst-case-optimal estimators for the FA frontier

Quantified finite-sample effects on group welfare

Provided confidence bands for fairness-accuracy tradeoffs

Abstract

We study fairness-accuracy tradeoffs when a single predictive model must serve multiple demographic groups. A useful tool for understanding this tradeoff is the fairness-accuracy (FA) Pareto frontier, which characterizes the set of models that cannot be improved in either fairness or accuracy without worsening the other. While characterizing the FA frontier requires full knowledge of the data distribution, we focus on the finite-sample regime, quantifying how well a designer can approximate any point on the frontier from limited data and bounding the worst-case gap. In particular, we derive worst-case-optimal estimators that depend on the designer's knowledge of the covariate distribution. For each estimator, we characterize how finite-sample effects asymmetrically impact each group's welfare and identify optimal sample allocation strategies. Finally, we provide uniform finite-sample bounds for the entire FA frontier, yielding confidence bands that quantify the reliability of welfare comparisons across alternative fairness-accuracy tradeoffs.