The Importance of Distrust in AI

TL;DR

This paper emphasizes the importance of balancing trust and distrust in AI systems to promote appropriate reliance, especially in high-stakes applications, by analyzing trust concepts and proposing a dual trust-distrust approach.

Contribution

It introduces the novel idea of incorporating distrust alongside trust to improve user reliance on AI, addressing issues of disuse and overtrust.

Findings

Distrust can mitigate overtrust in AI systems.

Balancing trust and distrust enhances appropriate reliance.

Analysis of trust concepts from multiple research fields.

Abstract

In recent years the use of Artificial Intelligence (AI) has become increasingly prevalent in a growing number of fields. As AI systems are being adopted in more high-stakes areas such as medicine and finance, ensuring that they are trustworthy is of increasing importance. A concern that is prominently addressed by the development and application of explainability methods, which are purported to increase trust from its users and wider society. While an increase in trust may be desirable, an analysis of literature from different research fields shows that an exclusive focus on increasing trust may not be warranted. Something which is well exemplified by the recent development in AI chatbots, which while highly coherent tend to make up facts. In this contribution, we investigate the concepts of trust, trustworthiness, and user reliance. In order to foster appropriate reliance on AI we need to prevent both disuse of these systems as well as overtrust. From our analysis of research on interpersonal trust, trust in automation, and trust in (X)AI, we identify the potential merit of the distinction between trust and distrust (in AI). We propose that alongside trust a healthy amount of distrust is of additional value for mitigating disuse and overtrust. We argue that by considering and evaluating both trust and distrust, we can ensure that users can rely appropriately on trustworthy AI, which can both be useful as well as fallible.