Fairness May Backfire: When Leveling-Down Occurs in Fair Machine Learning

TL;DR

This paper investigates when fairness constraints in machine learning improve or worsen outcomes for different groups, revealing that the effects depend on the decision-making regime and underlying data distribution.

Contribution

It provides a unified, distribution-free framework analyzing fairness impacts in binary classification, distinguishing between attribute-aware and attribute-blind regimes.

Findings

Fairness improves outcomes for disadvantaged groups in attribute-aware settings.

Fairness can harm or benefit groups in attribute-blind settings depending on data distribution.

Conditions for leveling up or leveling down are characterized, guiding fair ML deployment.

Abstract

As machine learning (ML) systems increasingly shape access to credit, jobs, and other opportunities, the fairness of algorithmic decisions has become a central concern. Yet it remains unclear when enforcing fairness constraints in these systems genuinely improves outcomes for affected groups or instead leads to "leveling down," making one or both groups worse off. We address this question in a unified, population-level (Bayes) framework for binary classification under prevalent group fairness notions. Our Bayes approach is distribution-free and algorithm-agnostic, isolating the intrinsic effect of fairness requirements from finite-sample noise and from training and intervention specifics. We analyze two deployment regimes for ML classifiers under common legal and governance constraints: attribute-aware decision-making (sensitive attributes available at decision time) and attribute-blind decision-making (sensitive attributes excluded from prediction). We show that, in the attribute-aware regime, fair ML necessarily (weakly) improves outcomes for the disadvantaged group and (weakly) worsens outcomes for the advantaged group. In contrast, in the attribute-blind regime, the impact of fairness is distribution-dependent: fairness can benefit or harm either group and may shift both groups' outcomes in the same direction, leading to either leveling up or leveling down. We characterize the conditions under which these patterns arise and highlight the role of "masked" candidates in driving them. Overall, our results provide structural guidance on when pursuing algorithmic fairness is likely to improve group outcomes and when it risks systemic leveling down, informing fair ML design and deployment choices.