Zombies in the Loop? Humans Trust Untrustworthy AI-Advisors for Ethical Decisions

TL;DR

This study reveals that people tend to overtrust AI ethical advisors even when they are untrustworthy or poorly understood, highlighting the need for improved digital literacy to promote responsible AI use.

Contribution

The paper demonstrates that users overtrust AI advisors in ethical decision-making regardless of their transparency or trustworthiness, challenging assumptions about AI trustworthiness.

Findings

Users trust untrustworthy AI advice in ethical dilemmas

Information about AI training data does not reduce trust

Digital literacy could mitigate overtrust in AI systems

Abstract

Departing from the claim that AI needs to be trustworthy, we find that ethical advice from an AI-powered algorithm is trusted even when its users know nothing about its training data and when they learn information about it that warrants distrust. We conducted online experiments where the subjects took the role of decision-makers who received advice from an algorithm on how to deal with an ethical dilemma. We manipulated the information about the algorithm and studied its influence. Our findings suggest that AI is overtrusted rather than distrusted. We suggest digital literacy as a potential remedy to ensure the responsible use of AI.