On the (im)possibility of fairness

TL;DR

This paper formalizes the concept of algorithmic fairness using a mathematical framework, revealing that different fairness notions require distinct assumptions about unobservable variables and their relation to observable data.

Contribution

It introduces a formal construct space to distinguish observable and unobservable variables, clarifying the assumptions needed for various fairness criteria.

Findings

Different fairness notions require different assumptions about the construct-to-decision mapping.

Formalization highlights the importance of explicitly stating assumptions about unobservable variables.

Framework aids in understanding the compatibility and limitations of fairness criteria.

Abstract

What does it mean for an algorithm to be fair? Different papers use different notions of algorithmic fairness, and although these appear internally consistent, they also seem mutually incompatible. We present a mathematical setting in which the distinctions in previous papers can be made formal. In addition to characterizing the spaces of inputs (the "observed" space) and outputs (the "decision" space), we introduce the notion of a construct space: a space that captures unobservable, but meaningful variables for the prediction. We show that in order to prove desirable properties of the entire decision-making process, different mechanisms for fairness require different assumptions about the nature of the mapping from construct space to decision space. The results in this paper imply that future treatments of algorithmic fairness should more explicitly state assumptions about the relationship between constructs and observations.