User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study

TL;DR

This comprehensive empirical study evaluates how different user groups understand AI explanations, revealing limitations in current XAI methods and emphasizing the need for user-centered design principles in explainability.

Contribution

It introduces a multidisciplinary, user-centered evaluation approach for XAI, providing new insights into explanation comprehensibility across diverse user expertise levels.

Findings

Current XAI methods have limitations in explanation comprehensibility.

User understanding varies significantly across expertise levels.

The study's methodology can be adapted to various data types and user profiles.

Abstract

This study is located in the Human-Centered Artificial Intelligence (HCAI) and focuses on the results of a user-centered assessment of commonly used eXplainable Artificial Intelligence (XAI) algorithms, specifically investigating how humans understand and interact with the explanations provided by these algorithms. To achieve this, we employed a multi-disciplinary approach that included state-of-the-art research methods from social sciences to measure the comprehensibility of explanations generated by a state-of-the-art lachine learning model, specifically the Gradient Boosting Classifier (XGBClassifier). We conducted an extensive empirical user study involving interviews with 39 participants from three different groups, each with varying expertise in data science, data visualization, and domain-specific knowledge related to the dataset used for training the machine learning model. Participants were asked a series of questions to assess their understanding of the model's explanations. To ensure replicability, we built the model using a publicly available dataset from the UC Irvine Machine Learning Repository, focusing on edible and non-edible mushrooms. Our findings reveal limitations in existing XAI methods and confirm the need for new design principles and evaluation techniques that address the specific information needs and user perspectives of different classes of AI stakeholders. We believe that the results of our research and the cross-disciplinary methodology we developed can be successfully adapted to various data types and user profiles, thus promoting dialogue and address opportunities in HCAI research. To support this, we are making the data resulting from our study publicly available.