Disparate Impact Diminishes Consumer Trust Even for Advantaged Users

TL;DR

This study shows that algorithmic unfairness, specifically disparate impact based on gender, reduces consumer trust in decision-support systems, regardless of whether users are advantaged or disadvantaged by the system.

Contribution

It provides empirical evidence that disparate impact diminishes trust in AI systems and affects all user groups equally, emphasizing fairness's role in consumer AI.

Findings

Disparate impact decreases consumer trust.

Trust reduction is similar across user groups.

Fairness is crucial for consumer AI acceptance.

Abstract

Systems aiming to aid consumers in their decision-making (e.g., by implementing persuasive techniques) are more likely to be effective when consumers trust them. However, recent research has demonstrated that the machine learning algorithms that often underlie such technology can act unfairly towards specific groups (e.g., by making more favorable predictions for men than for women). An undesired disparate impact resulting from this kind of algorithmic unfairness could diminish consumer trust and thereby undermine the purpose of the system. We studied this effect by conducting a between-subjects user study investigating how (gender-related) disparate impact affected consumer trust in an app designed to improve consumers' financial decision-making. Our results show that disparate impact decreased consumers' trust in the system and made them less likely to use it. Moreover, we find that trust was affected to the same degree across consumer groups (i.e., advantaged and disadvantaged users) despite both of these consumer groups recognizing their respective levels of personal benefit. Our findings highlight the importance of fairness in consumer-oriented artificial intelligence systems.