Study on the Helpfulness of Explainable Artificial Intelligence

TL;DR

This paper evaluates the effectiveness of explainable AI methods by measuring users' ability to perform proxy tasks, emphasizing human-centered assessment in critical applications.

Contribution

It introduces a user-centered evaluation approach for XAI, focusing on human decision-making and trust, supported by a comprehensive user study of state-of-the-art methods.

Findings

Differences in trust and skepticism across XAI methods.

Users' ability to judge AI decisions varies with explanation quality.

Proposed evaluation approach offers a more objective measure of XAI helpfulness.

Abstract

Explainable Artificial Intelligence (XAI) is essential for building advanced machine learning-powered applications, especially in critical domains such as medical diagnostics or autonomous driving. Legal, business, and ethical requirements motivate using effective XAI, but the increasing number of different methods makes it challenging to pick the right ones. Further, as explanations are highly context-dependent, measuring the effectiveness of XAI methods without users can only reveal a limited amount of information, excluding human factors such as the ability to understand it. We propose to evaluate XAI methods via the user's ability to successfully perform a proxy task, designed such that a good performance is an indicator for the explanation to provide helpful information. In other words, we address the helpfulness of XAI for human decision-making. Further, a user study on state-of-the-art methods was conducted, showing differences in their ability to generate trust and skepticism and the ability to judge the rightfulness of an AI decision correctly. Based on the results, we highly recommend using and extending this approach for more objective-based human-centered user studies to measure XAI performance in an end-to-end fashion.