Cases for Explainable Software Systems:Characteristics and Examples

TL;DR

This paper introduces a taxonomy and set of scenarios for understanding and benchmarking explanation needs in complex software systems, aiding research and development of explainable AI and interactive systems.

Contribution

It presents a structured taxonomy of explanation needs and concrete explanation cases to guide requirements and establish benchmarks for explainable systems.

Findings

A comprehensive taxonomy of explanation needs.

Concrete scenarios illustrating explanation demands.

A foundation for benchmarking explainable systems.

Abstract

The need for systems to explain behavior to users has become more evident with the rise of complex technology like machine learning or self-adaptation. In general, the need for an explanation arises when the behavior of a system does not match the user's expectations. However, there may be several reasons for a mismatch including errors, goal conflicts, or multi-agent interference. Given the various situations, we need precise and agreed descriptions of explanation needs as well as benchmarks to align research on explainable systems. In this paper, we present a taxonomy that structures needs for an explanation according to different reasons. We focus on explanations to improve the user interaction with the system. For each leaf node in the taxonomy, we provide a scenario that describes a concrete situation in which a software system should provide an explanation. These scenarios, called explanation cases, illustrate the different demands for explanations. Our taxonomy can guide the requirements elicitation for explanation capabilities of interactive intelligent systems and our explanation cases build the basis for a common benchmark. We are convinced that both, the taxonomy and the explanation cases, help the community to align future research on explainable systems.