An Actionability Assessment Tool for Explainable AI

TL;DR

This paper presents a new seven-question tool to assess the actionability of explanations in AI, helping improve human-centered algorithmic recourse by aligning assessments with human judgments.

Contribution

It introduces the first clear human-centered definition of actionability and develops an effective tool for its assessment in explainable AI.

Findings

The tool effectively discriminates between explanation types based on actionability.

Human judgments align with the tool's assessments of actionability.

Context influences actionability evaluations, indicating need for domain-specific adaptations.

Abstract

In this paper, we introduce and evaluate a tool for researchers and practitioners to assess the actionability of information provided to users to support algorithmic recourse. While there are clear benefits of recourse from the user's perspective, the notion of actionability in explainable AI research remains vague, and claims of `actionable' explainability techniques are based on the researchers' intuition. Inspired by definitions and instruments for assessing actionability in other domains, we construct a seven-question tool and evaluate its effectiveness through two user studies. We show that the tool discriminates actionability across explanation types and that the distinctions align with human judgements. We show the impact of context on actionability assessments, suggesting that domain-specific tool adaptations may foster more human-centred algorithmic systems. This is a significant step forward for research and practices into actionable explainability and algorithmic recourse, providing the first clear human-centred definition and tool for assessing actionability in explainable AI.