Identifiability of Causal-based Fairness Notions: A State of the Art

TL;DR

This paper reviews the major results on the identifiability of causality-based fairness notions in machine learning, highlighting their advantages over observational fairness measures and illustrating key concepts with examples.

Contribution

It compiles and explains the main identifiability results relevant for applying causality-based fairness in machine learning.

Findings

Causality-based fairness notions are more reliable against statistical anomalies.

Many causality-based fairness measures face an identifiability problem.

The paper provides illustrative examples and causal graphs for clarity.

Abstract

Machine learning algorithms can produce biased outcome/prediction, typically, against minorities and under-represented sub-populations. Therefore, fairness is emerging as an important requirement for the large scale application of machine learning based technologies. The most commonly used fairness notions (e.g. statistical parity, equalized odds, predictive parity, etc.) are observational and rely on mere correlation between variables. These notions fail to identify bias in case of statistical anomalies such as Simpson's or Berkson's paradoxes. Causality-based fairness notions (e.g. counterfactual fairness, no-proxy discrimination, etc.) are immune to such anomalies and hence more reliable to assess fairness. The problem of causality-based fairness notions, however, is that they are defined in terms of quantities (e.g. causal, counterfactual, and path-specific effects) that are not always measurable. This is known as the identifiability problem and is the topic of a large body of work in the causal inference literature. This paper is a compilation of the major identifiability results which are of particular relevance for machine learning fairness. The results are illustrated using a large number of examples and causal graphs. The paper would be of particular interest to fairness researchers, practitioners, and policy makers who are considering the use of causality-based fairness notions as it summarizes and illustrates the major identifiability results