Comparing Fairness Criteria Based on Social Outcome

TL;DR

This paper compares different fairness criteria in algorithmic decision-making, analyzing their impact on social outcomes and disparities, and finds that equalized odds uniquely eliminates group-level disparities.

Contribution

It provides a comparative analysis of fairness notions like CB, DP, and EO, highlighting EO's effectiveness in removing group disparities.

Findings

EO removes group-level disparity

Empirical analysis of social welfare impacts

Comparison of fairness criteria effects

Abstract

Fairness in algorithmic decision-making processes is attracting increasing concern. When an algorithm is applied to human-related decision-making an estimator solely optimizing its predictive power can learn biases on the existing data, which motivates us the notion of fairness in machine learning. while several different notions are studied in the literature, little studies are done on how these notions affect the individuals. We demonstrate such a comparison between several policies induced by well-known fairness criteria, including the color-blind (CB), the demographic parity (DP), and the equalized odds (EO). We show that the EO is the only criterion among them that removes group-level disparity. Empirical studies on the social welfare and disparity of these policies are conducted.