Fairness Under Composition

TL;DR

This paper studies how fairness properties of individual algorithms behave when combined into systems, revealing pitfalls of naive composition and proposing methods for constructing fair composite systems.

Contribution

It introduces the concept of fairness under composition, identifies pitfalls of naive approaches, and provides constructions for fair systems from individually fair classifiers.

Findings

Naive composition can lead to unfair systems despite fair individual classifiers.

Careful combination of unfair components can produce fair systems.

Group fairness definitions may give misleading signals when systems are composed.

Abstract

Algorithmic fairness, and in particular the fairness of scoring and classification algorithms, has become a topic of increasing social concern and has recently witnessed an explosion of research in theoretical computer science, machine learning, statistics, the social sciences, and law. Much of the literature considers the case of a single classifier (or scoring function) used once, in isolation. In this work, we initiate the study of the fairness properties of systems composed of algorithms that are fair in isolation; that is, we study fairness under composition. We identify pitfalls of naive composition and give general constructions for fair composition, demonstrating both that classifiers that are fair in isolation do not necessarily compose into fair systems and also that seemingly unfair components may be carefully combined to construct fair systems. We focus primarily on the individual fairness setting proposed in [Dwork, Hardt, Pitassi, Reingold, Zemel, 2011], but also extend our results to a large class of group fairness definitions popular in the recent literature, exhibiting several cases in which group fairness definitions give misleading signals under composition.