Fairness vs Performance: Characterizing the Pareto Frontier of Algorithmic Decision Systems

TL;DR

This paper characterizes the trade-offs between fairness and performance in algorithmic decision systems by analyzing the Pareto frontier of optimal decision rules across various fairness metrics and utility functions.

Contribution

It provides a comprehensive analysis of the Pareto-optimal decision rules, extending existing theorems to generalized fairness metrics and connecting formal fairness with legal and ethical considerations.

Findings

Pareto frontier consists of deterministic, group-specific threshold rules.

Depending on the fairness metric, the frontier may include upper-bound threshold rules.

The location of the Pareto frontier depends only on population, utility, and fairness score, not on algorithm design.

Abstract

Designing fair algorithmic decision systems requires balancing model performance with fairness toward affected individuals: More fairness might require sacrificing some performance and vice versa, yet the space of possible trade-offs is still poorly understood. We investigate fairness in binary prediction-based decision problems by conceptualizing decision making as a multi-objective optimization problem that simultaneously considers decision-maker utility and group fairness. We investigate the set of Pareto-optimal decision rules for arbitrary utility functions for decision maker, arbitrary population distributions, and a wide range of group fairness metrics. We find that the Pareto frontier consists of deterministic, group-specific threshold rules applied to individuals' success probability. This complements existing optimality theorems from literature which, for specific fairness constraints, posit lower-bound threshold rules only. However we also show that, depending on the used fairness metric, the Pareto frontier may include upper-bound threshold rules, thus preferring individuals with lower success probabilities. We show that the location of the Pareto frontier depends only on population characteristics, utility functions and fairness score, but not on the technical design of the algorithm - our findings hold for pre-, in-, and post-processing approaches alike. Our results generalize existing optimality theorems for fairness-constrained classification and extend them to generalized fairness metrics and fairness principles, and to partial fairness regimes. This paper connects formal fairness research with legal and ethical requirements to search for less discriminatory alternatives, offering a principled foundation for evaluating and comparing algorithmic decision systems.