Influence of the algorithm's reliability and transparency in the user's decision-making process

TL;DR

This study investigates how algorithm transparency and reliability influence user decision-making and confidence, revealing that users maintain moderate confidence even with unreliable algorithms but do not blindly trust wrong decisions.

Contribution

It provides empirical evidence on user trust and decision-making behavior in response to varying levels of algorithm transparency and reliability.

Findings

Users maintain moderate confidence despite low reliability.

Participants do not blindly follow incorrect algorithm decisions.

Transparency influences user trust and decision confidence.

Abstract

Algorithms have been becoming increasingly relevant for various decision-making processes in the forms of Decision Support Systems or Decision-making systems in areas such as Criminal-Justice systems, Job Application Filtering, Medicine, and Healthcare to name a few. It is crucial for these algorithms to be fair and for the users to have confidence in these decisions, especially in the above contexts, because they have a high impact on society. We conduct an online empirical study with 61 participants to find out how the change in transparency and reliability of an algorithm which determines the probability of lesions being melanoma could impact users' decision-making process, as well as the confidence in the decisions made by the algorithm. The results indicate that people show at least moderate confidence in the decisions of the algorithm even when the reliability is bad. However, they would not blindly follow the algorithm's wrong decisions.