Helpful, Misleading or Confusing: How Humans Perceive Fundamental Building Blocks of Artificial Intelligence Explanations

TL;DR

This paper investigates how different human stakeholders perceive the comprehensibility of fundamental AI explanation formats, aiming to develop evaluation methods for explainability in simple decision models.

Contribution

It introduces a framework for assessing human perceptions of various AI explanation representations and proposes a methodology for evaluating explainability of basic decision models.

Findings

Stakeholders perceive explanation formats differently based on their background.

A new evaluation framework for explainability of simple models is proposed.

Guidelines for conducting user studies on AI explanations are outlined.

Abstract

Explainable artificial intelligence techniques are developed at breakneck speed, but suitable evaluation approaches lag behind. With explainers becoming increasingly complex and a lack of consensus on how to assess their utility, it is challenging to judge the benefit and effectiveness of different explanations. To address this gap, we take a step back from sophisticated predictive algorithms and instead look into explainability of simple decision-making models. In this setting, we aim to assess how people perceive comprehensibility of their different representations such as mathematical formulation, graphical representation and textual summarisation (of varying complexity and scope). This allows us to capture how diverse stakeholders -- engineers, researchers, consumers, regulators and the like -- judge intelligibility of fundamental concepts that more elaborate artificial intelligence explanations are built from. This position paper charts our approach to establishing appropriate evaluation methodology as well as a conceptual and practical framework to facilitate setting up and executing relevant user studies.