Reassessing Evaluation Functions in Algorithmic Recourse: An Empirical Study from a Human-Centered Perspective

TL;DR

This study empirically investigates whether minimizing the recourse distance in AI-generated suggestions truly influences human acceptance and action, revealing that acceptance is not solely determined by proximity to the desired state.

Contribution

It provides the first empirical evidence questioning the core assumption of recourse acceptance based on distance minimization, highlighting the need for human-centered evaluation.

Findings

Acceptance of recourses was not correlated with recourse distance.

Willingness to act peaked at minimal recourse distance.

Participants' acceptance remained constant beyond the minimal distance.

Abstract

In this study, we critically examine the foundational premise of algorithmic recourse - a process of generating counterfactual action plans (i.e., recourses) assisting individuals to reverse adverse decisions made by AI systems. The assumption underlying algorithmic recourse is that individuals accept and act on recourses that minimize the gap between their current and desired states. This assumption, however, remains empirically unverified. To address this issue, we conducted a user study with 362 participants and assessed whether minimizing the distance function, a metric of the gap between the current and desired states, indeed prompts them to accept and act upon suggested recourses. Our findings reveal a nuanced landscape: participants' acceptance of recourses did not correlate with the recourse distance. Moreover, participants' willingness to act upon recourses peaked at the minimal recourse distance but was otherwise constant. These findings cast doubt on the prevailing assumption of algorithmic recourse research and signal the need to rethink the evaluation functions to pave the way for human-centered recourse generation.