Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks

TL;DR

This survey categorizes XAI studies for data analysis tasks using a framework based on what, why, and who, aiming to improve understanding, comparability, and reporting standards in the field.

Contribution

It introduces a novel categorization method for XAI studies and proposes guidelines for better study design and reporting to address current gaps.

Findings

Identified key issues like inadequate task descriptions and lack of user context.

Proposed a three-dimensional framework for categorizing XAI studies.

Suggested reporting guidelines to enhance study comparability and generalizability.

Abstract

Research into explainable artificial intelligence (XAI) for data analysis tasks suffer from a large number of contradictions and lack of concrete design recommendations stemming from gaps in understanding the tasks that require AI assistance. In this paper, we drew on multiple fields such as visual analytics, cognition, and dashboard design to propose a method for categorising and comparing XAI studies under three dimensions: what, why, and who. We identified the main problems as: inadequate descriptions of tasks, context-free studies, and insufficient testing with target users. We propose that studies should specifically report on their users' domain, AI, and data analysis expertise to illustrate the generalisability of their findings. We also propose study guidelines for designing and reporting XAI tasks to improve the XAI community's ability to parse the rapidly growing field. We hope that our contribution can help researchers and designers better identify which studies are most relevant to their work, what gaps exist in the research, and how to handle contradictory results regarding XAI design.