Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI

TL;DR

This paper develops a formal model of human trust in AI, focusing on prerequisites, causes, and goals, to improve AI design, evaluation, and the connection with explainable AI.

Contribution

It introduces a formal trust model inspired by sociology, including concepts of contractual trust and trustworthiness, to better understand and promote warranted trust in AI systems.

Findings

Formalizes trust as vulnerability and impact anticipation

Defines 'contractual trust' and 'trustworthiness' in AI context

Discusses methods to design and evaluate trustworthy AI

Abstract

Trust is a central component of the interaction between people and AI, in that 'incorrect' levels of trust may cause misuse, abuse or disuse of the technology. But what, precisely, is the nature of trust in AI? What are the prerequisites and goals of the cognitive mechanism of trust, and how can we promote them, or assess whether they are being satisfied in a given interaction? This work aims to answer these questions. We discuss a model of trust inspired by, but not identical to, sociology's interpersonal trust (i.e., trust between people). This model rests on two key properties of the vulnerability of the user and the ability to anticipate the impact of the AI model's decisions. We incorporate a formalization of 'contractual trust', such that trust between a user and an AI is trust that some implicit or explicit contract will hold, and a formalization of 'trustworthiness' (which detaches from the notion of trustworthiness in sociology), and with it concepts of 'warranted' and 'unwarranted' trust. We then present the possible causes of warranted trust as intrinsic reasoning and extrinsic behavior, and discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted. Finally, we elucidate the connection between trust and XAI using our formalization.