Fairness Through Counterfactual Utilities

TL;DR

This paper introduces a unified, counterfactual utility-based fairness framework applicable across all machine learning environments, addressing limitations of existing definitions by focusing on utilities and counterfactual outcomes.

Contribution

It provides a generalized set of fairness definitions that extend beyond classification to all ML settings, unifying previous interpretations and resolving known fairness issues.

Findings

Framework unifies fairness definitions across ML environments

Counterfactual utility approach prevents fairness loopholes

Many existing fairness notions are special cases of this framework

Abstract

Group fairness definitions such as Demographic Parity and Equal Opportunity make assumptions about the underlying decision-problem that restrict them to classification problems. Prior work has translated these definitions to other machine learning environments, such as unsupervised learning and reinforcement learning, by implementing their closest mathematical equivalent. As a result, there are numerous bespoke interpretations of these definitions. Instead, we provide a generalized set of group fairness definitions that unambiguously extend to all machine learning environments while still retaining their original fairness notions. We derive two fairness principles that enable such a generalized framework. First, our framework measures outcomes in terms of utilities, rather than predictions, and does so for both the decision-algorithm and the individual. Second, our framework considers counterfactual outcomes, rather than just observed outcomes, thus preventing loopholes where fairness criteria are satisfied through self-fulfilling prophecies. We provide concrete examples of how our counterfactual utility fairness framework resolves known fairness issues in classification, clustering, and reinforcement learning problems. We also show that many of the bespoke interpretations of Demographic Parity and Equal Opportunity fit nicely as special cases of our framework.