On the Relation of Trust and Explainability: Why to Engineer for Trustworthiness

TL;DR

This paper explores the complex relationship between explainability and trust in software systems, emphasizing the importance of engineering for trustworthiness rather than trust alone, with explainability enhancing trustworthiness.

Contribution

It argues for focusing on trustworthiness in system design, highlighting how explainability can contribute to trustworthiness rather than directly fostering trust.

Findings

Explainability does not necessarily increase trust.

Trustworthiness can be engineered independently of trust.

Explainability can enhance a system's trustworthiness.

Abstract

Recently, requirements for the explainability of software systems have gained prominence. One of the primary motivators for such requirements is that explainability is expected to facilitate stakeholders' trust in a system. Although this seems intuitively appealing, recent psychological studies indicate that explanations do not necessarily facilitate trust. Thus, explainability requirements might not be suitable for promoting trust. One way to accommodate this finding is, we suggest, to focus on trustworthiness instead of trust. While these two may come apart, we ideally want both: a trustworthy system and the stakeholder's trust. In this paper, we argue that even though trustworthiness does not automatically lead to trust, there are several reasons to engineer primarily for trustworthiness -- and that a system's explainability can crucially contribute to its trustworthiness.