Towards Human-centered Design of Explainable Artificial Intelligence (XAI): A Survey of Empirical Studies

TL;DR

This survey reviews empirical studies on human-centered explainable AI, highlighting design approaches, stakeholder needs, evaluation metrics, challenges, and proposing a framework for future research.

Contribution

It systematically analyzes existing empirical research in human-centered XAI and introduces a comprehensive framework for designing and evaluating XAI with human factors in mind.

Findings

Stakeholders' needs vary across domains and tasks.

Current XAI evaluation metrics often focus on technical aspects.

Identified challenges include balancing explainability and performance.

Abstract

With the advances of AI research, AI has been increasingly adopted in numerous domains, ranging from low-stakes daily tasks such as movie recommendations to high-stakes tasks such as medicine, and criminal justice decision-making. Explainability is becoming an essential requirement for people to understand, trust and adopt AI applications. Despite a vast collection of explainable AI (XAI) algorithms produced by the AI research community, successful examples of XAI are still relatively scarce in real-world AI applications. This can be due to the gap between what the XAI is designed for and how the XAI is actually perceived by end-users. As explainability is an inherently human-centered property, in recent years, the XAI field is starting to embrace human-centered approaches and increasingly realizing the importance of empirical studies of XAI design by involving human subjects. To move a step towards a systematic review of empirical study for human-centered XAI design, in this survey, we first brief the technical landscape of commonly used XAI algorithms in existing empirical studies. Then we analyze the diverse stakeholders and needs-finding approaches. Next, we provide an overview of the design space explored in the current human-centered XAI design. Further, we summarize the evaluation metrics based on evaluation goals. Afterward, we analyze the common findings and pitfalls derived from existing studies. For each chapter, we provide a summary of current challenges and research opportunities. Finally, we conclude the survey with a framework for human-centered XAI design with empirical studies.