Identifying Explanation Needs: Towards a Catalog of User-based Indicators

TL;DR

This paper develops a catalog of 39 user-based indicators derived from user behavior, system events, and emotional states to identify when explanations are needed in complex digital systems.

Contribution

It introduces a novel set of runtime indicators for detecting explanation needs, facilitating personalized and timely explanations in software systems.

Findings

Catalog includes 17 user behavior indicators

Catalog contains 8 system event indicators

Includes 14 emotional and physical reaction indicators

Abstract

In today's digitalized world, where software systems are becoming increasingly ubiquitous and complex, the quality aspect of explainability is gaining relevance. A major challenge in achieving adequate explanations is the elicitation of individual explanation needs, as it may be subject to severe hypothetical or confirmation biases. To address these challenges, we aim to establish user-based indicators concerning user behavior or system events that can be captured at runtime to determine when a need for explanations arises. In this work, we conducted explorative research in form of an online study to collect self-reported indicators that could indicate a need for explanation. We compiled a catalog containing 17 relevant indicators concerning user behavior, 8 indicators concerning system events and 14 indicators concerning emotional states or physical reactions. We also analyze the relationships between these indicators and different types of need for explanation. The established indicators can be used in the elicitation process through prototypes, as well as after publication to gather requirements from already deployed applications using telemetry and usage data. Moreover, these indicators can be used to trigger explanations at appropriate moments during the runtime.