Distributive Justice as the Foundational Premise of Fair ML: Unification, Extension, and Interpretation of Group Fairness Metrics

TL;DR

This paper develops a comprehensive framework linking group fairness metrics in machine learning to theories of distributive justice, clarifying their moral implications and expanding their scope to address existing criticisms.

Contribution

It unifies and interprets group fairness metrics through the lens of distributive justice, revealing their normative assumptions and proposing extensions to improve fairness evaluations.

Findings

Reveals normative choices behind standard group fairness metrics

Provides a framework to interpret their moral significance

Suggests extensions to address criticisms of parity-based fairness

Abstract

Group fairness metrics are an established way of assessing the fairness of prediction-based decision-making systems. However, these metrics are still insufficiently linked to philosophical theories, and their moral meaning is often unclear. In this paper, we propose a comprehensive framework for group fairness metrics, which links them to more theories of distributive justice. The different group fairness metrics differ in their choices about how to measure the benefit or harm of a decision for the affected individuals, and what moral claims to benefits are assumed. Our unifying framework reveals the normative choices associated with standard group fairness metrics and allows an interpretation of their moral substance. In addition, this broader view provides a structure for the expansion of standard fairness metrics that we find in the literature. This expansion allows addressing several criticisms of standard group fairness metrics, specifically: (1) they are parity-based, i.e., they demand some form of equality between groups, which may sometimes be detrimental to marginalized groups; (2) they only compare decisions across groups but not the resulting consequences for these groups; and (3) the full breadth of the distributive justice literature is not sufficiently represented.