Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems

TL;DR

This paper explores how explanations of automated decision systems influence users' perceptions of fairness, emphasizing the importance of perceptions aligning with the system's actual fairness.

Contribution

It introduces the concept of appropriate fairness perceptions and proposes a new study design to evaluate explanation effectiveness in assessing system fairness.

Findings

Perceptions should increase only if the system is fair.

Explanations can reveal system shortcomings when unfair.

Framework for evaluating explanation effectiveness in fairness assessment.

Abstract

It is often argued that one goal of explaining automated decision systems (ADS) is to facilitate positive perceptions (e.g., fairness or trustworthiness) of users towards such systems. This viewpoint, however, makes the implicit assumption that a given ADS is fair and trustworthy, to begin with. If the ADS issues unfair outcomes, then one might expect that explanations regarding the system's workings will reveal its shortcomings and, hence, lead to a decrease in fairness perceptions. Consequently, we suggest that it is more meaningful to evaluate explanations against their effectiveness in enabling people to appropriately assess the quality (e.g., fairness) of an associated ADS. We argue that for an effective explanation, perceptions of fairness should increase if and only if the underlying ADS is fair. In this in-progress work, we introduce the desideratum of appropriate fairness perceptions, propose a novel study design for evaluating it, and outline next steps towards a comprehensive experiment.