Understanding Decision Subjects' Engagement with and Perceived Fairness of AI Models When Opportunities of Qualification Improvement Exist

TL;DR

This study investigates how decision subjects' engagement and fairness perceptions are influenced by AI model fairness, considering strategic responses like continued interaction and self-improvement, through three human-subject experiments.

Contribution

It reveals that decision subjects' willingness to engage or improve does not change with model fairness, but fairness perceptions are affected by systematic biases and difficulty of self-improvement.

Findings

Model fairness does not affect willingness to interact or improve.

Subjects perceive unfair models as less fair, especially with larger qualification improvement challenges.

Perceived fairness is influenced by bias against protected groups.

Abstract

We explore how an AI model's decision fairness affects people's engagement with and perceived fairness of the model if they are subject to its decisions, but could repeatedly and strategically respond to these decisions. Two types of strategic responses are considered -- people could determine whether to continue interacting with the model, and whether to invest in themselves to improve their chance of future favorable decisions from the model. Via three human-subject experiments, we found that in decision subjects' strategic, repeated interactions with an AI model, the model's decision fairness does not change their willingness to interact with the model or to improve themselves, even when the model exhibits unfairness on salient protected attributes. However, decision subjects still perceive the AI model to be less fair when it systematically biases against their group, especially if the difficulty of improving one's qualification for the favorable decision is larger for the lowly-qualified people.