Identifying the desert decision rule to assess and achieve fairness

TL;DR

This paper introduces the concept of desert decision to improve fairness in decision-making by focusing on the fair underlying decision rather than biased observed decisions.

Contribution

It proposes a novel framework for identifying and estimating desert decisions, linking fairness to measurement error models and providing robustness tools.

Findings

Established nonparametric identification under causal assumptions.

Developed estimators for desert decision rules and unfairness degree.

Proposed sensitivity analysis for robustness to assumption violations.

Abstract

We study fairness in decision-making when the data may encode systematic bias. Existing approaches typically impose fairness constraints while predicting the observed decision, which may itself be unfair. We propose a novel framework for characterising and addressing fairness issues by introducing the notion of desert decision, a latent variable representing the decision an individual rightfully deserves based on their actions, efforts, or abilities. This formulation shifts the prediction target from the potentially biased observed decision to the desert decision. We advocate achieving fair decision-making by predicting the desert decision and assessing unfairness by the discrepancy between desert and observed decisions. We establish nonparametric identification results under causally interpretable assumptions on the fairness of the desert decision and the unfairness mechanism of the observed decision. For estimation, we develop a sieve maximum likelihood estimator for the desert decision rule and an influence-function-based estimator for the degree of unfairness. Sensitivity analysis procedures are further proposed to assess robustness to violations of identifying assumptions. Our framework connects fairness with measurement error models, aligning predictive accuracy with fairness relative to an appropriate target, and providing a structural approach to modelling the unfairness mechanism.