Stakeholders in Explainable AI

TL;DR

This paper explores the diverse stakeholder perspectives on explainability in AI, highlighting differing needs and interpretations to clarify what stakeholders mean when they ask 'Why?' of an AI system.

Contribution

It dissects the varied stakeholder communities' views on explainability using software engineering and epistemological distinctions, clarifying their overlapping and diverging concerns.

Findings

Stakeholder communities have distinct interpretability needs.

Differences in 'validation' and 'verification' concerns influence explainability.

Clarification helps align stakeholder expectations with AI explanations.

Abstract

There is general consensus that it is important for artificial intelligence (AI) and machine learning systems to be explainable and/or interpretable. However, there is no general consensus over what is meant by 'explainable' and 'interpretable'. In this paper, we argue that this lack of consensus is due to there being several distinct stakeholder communities. We note that, while the concerns of the individual communities are broadly compatible, they are not identical, which gives rise to different intents and requirements for explainability/interpretability. We use the software engineering distinction between validation and verification, and the epistemological distinctions between knowns/unknowns, to tease apart the concerns of the stakeholder communities and highlight the areas where their foci overlap or diverge. It is not the purpose of the authors of this paper to 'take sides' - we count ourselves as members, to varying degrees, of multiple communities - but rather to help disambiguate what stakeholders mean when they ask 'Why?' of an AI.