How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness

TL;DR

This study examines public perceptions of different algorithmic fairness definitions in loan decisions, revealing preferences for calibrated fairness and support for affirmative action principles.

Contribution

It provides empirical insights into how ordinary people perceive various fairness definitions and how sensitive information influences these perceptions.

Findings

Calibrated fairness is generally perceived as the fairest.

Public support for affirmative action is observed.

Fairness perceptions are affected by the inclusion of sensitive information.

Abstract

What is the best way to define algorithmic fairness? While many definitions of fairness have been proposed in the computer science literature, there is no clear agreement over a particular definition. In this work, we investigate ordinary people's perceptions of three of these fairness definitions. Across two online experiments, we test which definitions people perceive to be the fairest in the context of loan decisions, and whether fairness perceptions change with the addition of sensitive information (i.e., race of the loan applicants). Overall, one definition (calibrated fairness) tends to be more preferred than the others, and the results also provide support for the principle of affirmative action.