Who Gets What, According to Whom? An Analysis of Fairness Perceptions in Service Allocation

TL;DR

This study investigates how perceptions of fairness in service allocation vary based on context, framing, and respondent identity, highlighting complexities in integrating fairness perceptions into machine learning systems.

Contribution

It introduces a multi-factor conjoint analysis to quantify how context, framing, and respondent identity influence fairness perceptions, revealing nuanced insights for algorithmic fairness design.

Findings

Perceptions of fairness are significantly affected by context and framing.

Who answers the question influences fairness judgments in complex ways.

Results challenge existing theoretical models of fairness perception.

Abstract

Algorithmic fairness research has traditionally been linked to the disciplines of philosophy, ethics, and economics, where notions of fairness are prescriptive and seek objectivity. Increasingly, however, scholars are turning to the study of what different people perceive to be fair, and how these perceptions can or should help to shape the design of machine learning, particularly in the policy realm. The present work experimentally explores five novel research questions at the intersection of the "Who," "What," and "How" of fairness perceptions. Specifically, we present the results of a multi-factor conjoint analysis study that quantifies the effects of the specific context in which a question is asked, the framing of the given question, and who is answering it. Our results broadly suggest that the "Who" and "What," at least, matter in ways that are 1) not easily explained by any one theoretical perspective, 2) have critical implications for how perceptions of fairness should be measured and/or integrated into algorithmic decision-making systems.