Designing Equitable Algorithms

TL;DR

This paper critically examines common fairness constraints in predictive algorithms, revealing they can worsen outcomes for marginalized groups and emphasizing the need for a broader understanding of algorithmic fairness.

Contribution

It demonstrates that standard fairness constraints may harm marginalized groups and proposes new considerations for designing equitable algorithms.

Findings

Fairness constraints can worsen outcomes for marginalized groups.

Trade-offs exist between fairness constraints and welfare improvements.

Recommendations for more equitable algorithm design.

Abstract

Predictive algorithms are now used to help distribute a large share of our society's resources and sanctions, such as healthcare, loans, criminal detentions, and tax audits. Under the right circumstances, these algorithms can improve the efficiency and equity of decision-making. At the same time, there is a danger that the algorithms themselves could entrench and exacerbate disparities, particularly along racial, ethnic, and gender lines. To help ensure their fairness, many researchers suggest that algorithms be subject to at least one of three constraints: (1) no use of legally protected features, such as race, ethnicity, and gender; (2) equal rates of "positive" decisions across groups; and (3) equal error rates across groups. Here we show that these constraints, while intuitively appealing, often worsen outcomes for individuals in marginalized groups, and can even leave all groups worse off. The inherent trade-off we identify between formal fairness constraints and welfare improvements -- particularly for the marginalized -- highlights the need for a more robust discussion on what it means for an algorithm to be "fair". We illustrate these ideas with examples from healthcare and the criminal-legal system, and make several proposals to help practitioners design more equitable algorithms.