Defining Explainable AI for Requirements Analysis

TL;DR

This paper proposes a framework with three dimensions—Source, Depth, and Scope—to categorize explanatory requirements of various AI applications and match them with suitable ML techniques, focusing on requirements for trust and transparency.

Contribution

It introduces a novel three-dimensional framework for defining and matching explainability requirements in AI applications, emphasizing application-specific needs.

Findings

Three dimensions for categorizing explanations: Source, Depth, and Scope.

A methodology for aligning ML capabilities with application-specific explainability needs.

Focus on requirements-driven explanation matching rather than general explanation techniques.

Abstract

Explainable Artificial Intelligence (XAI) has become popular in the last few years. The Artificial Intelligence (AI) community in general, and the Machine Learning (ML) community in particular, is coming to the realisation that in many applications, for AI to be trusted, it must not only demonstrate good performance in its decisionmaking, but it also must explain these decisions and convince us that it is making the decisions for the right reasons. However, different applications have different requirements on the information required of the underlying AI system in order to convince us that it is worthy of our trust. How do we define these requirements? In this paper, we present three dimensions for categorising the explanatory requirements of different applications. These are Source, Depth and Scope. We focus on the problem of matching up the explanatory requirements of different applications with the capabilities of underlying ML techniques to provide them. We deliberately avoid including aspects of explanation that are already well-covered by the existing literature and we focus our discussion on ML although the principles apply to AI more broadly.