The Unfairness of $\varepsilon$-Fairness

TL;DR

This paper critiques $psilon$-fairness by showing it can lead to unfair real-world outcomes and advocates for a utility-based approach that considers actual impacts, demonstrated through college and credit examples.

Contribution

It introduces a utility-based fairness evaluation framework and demonstrates its importance over traditional probabilistic metrics in real-world decision contexts.

Findings

$psilon$-fairness can cause maximally unfair outcomes in practice.

Utility-based evaluation reveals necessary actions for true fairness.

Enhancing completion rates is key for fairness in college admissions.

Abstract

Fairness in decision-making processes is often quantified using probabilistic metrics. However, these metrics may not fully capture the real-world consequences of unfairness. In this article, we adopt a utility-based approach to more accurately measure the real-world impacts of decision-making process. In particular, we show that if the concept of $\varepsilon$-fairness is employed, it can possibly lead to outcomes that are maximally unfair in the real-world context. Additionally, we address the common issue of unavailable data on false negatives by proposing a reduced setting that still captures essential fairness considerations. We illustrate our findings with two real-world examples: college admissions and credit risk assessment. Our analysis reveals that while traditional probability-based evaluations might suggest fairness, a utility-based approach uncovers the necessary actions to truly achieve equality. For instance, in the college admission case, we find that enhancing completion rates is crucial for ensuring fairness. Summarizing, this paper highlights the importance of considering the real-world context when evaluating fairness.