Explaining Classifications to Non Experts: An XAI User Study of Post Hoc Explanations for a Classifier When People Lack Expertise

TL;DR

This study investigates how user expertise influences understanding of post-hoc explanations for a deep learning classifier, revealing significant differences based on domain familiarity and expertise level.

Contribution

It provides novel empirical insights into how domain expertise affects interpretation of explanations in XAI, especially across familiar and unfamiliar domains.

Findings

Understanding of explanations varies with domain familiarity.

Response times and perceptions differ based on expertise.

Expertise impacts perceived correctness and helpfulness.

Abstract

Very few eXplainable AI (XAI) studies consider how users understanding of explanations might change depending on whether they know more or less about the to be explained domain (i.e., whether they differ in their expertise). Yet, expertise is a critical facet of most high stakes, human decision making (e.g., understanding how a trainee doctor differs from an experienced consultant). Accordingly, this paper reports a novel, user study (N=96) on how peoples expertise in a domain affects their understanding of post-hoc explanations by example for a deep-learning, black box classifier. The results show that peoples understanding of explanations for correct and incorrect classifications changes dramatically, on several dimensions (e.g., response times, perceptions of correctness and helpfulness), when the image-based domain considered is familiar (i.e., MNIST) as opposed to unfamiliar (i.e., Kannada MNIST). The wider implications of these new findings for XAI strategies are discussed.