Machine learning fairness notions: Bridging the gap with real-world applications

TL;DR

This survey explores various fairness notions in machine learning, illustrating their differences through examples, and provides a decision diagram to help practitioners select the most appropriate fairness approach for specific real-world scenarios.

Contribution

It uniquely analyzes the suitability of different fairness notions for real-world applications and offers a decision diagram for practical decision-making.

Findings

Different fairness notions have distinct behaviors in real scenarios.

A decision diagram helps identify the most suitable fairness notion.

The survey bridges theoretical fairness concepts with practical applications.

Abstract

Fairness emerged as an important requirement to guarantee that Machine Learning (ML) predictive systems do not discriminate against specific individuals or entire sub-populations, in particular, minorities. Given the inherent subjectivity of viewing the concept of fairness, several notions of fairness have been introduced in the literature. This paper is a survey that illustrates the subtleties between fairness notions through a large number of examples and scenarios. In addition, unlike other surveys in the literature, it addresses the question of: which notion of fairness is most suited to a given real-world scenario and why? Our attempt to answer this question consists in (1) identifying the set of fairness-related characteristics of the real-world scenario at hand, (2) analyzing the behavior of each fairness notion, and then (3) fitting these two elements to recommend the most suitable fairness notion in every specific setup. The results are summarized in a decision diagram that can be used by practitioners and policymakers to navigate the relatively large catalog of ML.