Optimization Hierarchy for Fair Statistical Decision Problems

TL;DR

This paper introduces an optimization hierarchy for fair statistical decision problems, providing a systematic, theoretically grounded framework applicable across various decision-making tasks to ensure fairness through statistical independence.

Contribution

It develops a unified, systematic hierarchy of optimization problems for fairness, grounded in statistical decision theory, enabling consistent enforcement of fairness constraints.

Findings

Hierarchy converges to fairness constraints asymptotically

Numerical experiments demonstrate effectiveness across datasets

Applied to automated morphine dosing for fairness

Abstract

Data-driven decision-making has drawn scrutiny from policy makers due to fears of potential discrimination, and a growing literature has begun to develop fair statistical techniques. However, these techniques are often specialized to one model context and based on ad-hoc arguments, which makes it difficult to perform theoretical analysis. This paper develops an optimization hierarchy, which is a sequence of optimization problems with an increasing number of constraints, for fair statistical decision problems. Because our hierarchy is based on the framework of statistical decision problems, this means it provides a systematic approach for developing and studying fair versions of hypothesis testing, decision-making, estimation, regression, and classification. We use the insight that qualitative definitions of fairness are equivalent to statistical independence between the output of a statistical technique and a random variable that measures attributes for which fairness is desired. We use this insight to construct an optimization hierarchy that lends itself to numerical computation, and we use tools from variational analysis and random set theory to prove that higher levels of this hierarchy lead to consistency in the sense that it asymptotically imposes this independence as a constraint in corresponding statistical decision problems. We demonstrate numerical effectiveness of our hierarchy using several data sets, and we use our hierarchy to fairly perform automated dosing of morphine.