Navigating Fairness Measures and Trade-Offs

TL;DR

This paper proposes a theory based on Rawls' justice to guide the selection of fairness measures in AI, balancing fairness and accuracy while prioritizing vulnerable groups.

Contribution

It introduces a principled framework using Rawls' theory to navigate fairness measures and trade-offs in AI systems.

Findings

Framework prioritizes vulnerable groups in fairness decisions

Aligns philosophical justice with practical fairness measures

Provides a basis for operationalizing fairness in AI

Abstract

In order to monitor and prevent bias in AI systems we can use a wide range of (statistical) fairness measures. However, it is mathematically impossible to optimize for all of these measures at the same time. In addition, optimizing a fairness measure often greatly reduces the accuracy of the system (Kozodoi et al, 2022). As a result, we need a substantive theory that informs us how to make these decisions and for what reasons. I show that by using Rawls' notion of justice as fairness, we can create a basis for navigating fairness measures and the accuracy trade-off. In particular, this leads to a principled choice focusing on both the most vulnerable groups and the type of fairness measure that has the biggest impact on that group. This also helps to close part of the gap between philosophical accounts of distributive justice and the fairness literature that has been observed (Kuppler et al, 2021) and to operationalise the value of fairness.