Exploring Practitioner Perspectives On Training Data Attribution Explanations

TL;DR

This study explores how practitioners perceive training data attribution explanations in XAI, revealing their limited awareness and emphasizing the need for usability-focused evaluation to improve human-model collaboration.

Contribution

It provides empirical insights into practitioner perspectives on TDA explanations, highlighting gaps in awareness and suggesting directions for practical utility and evaluation improvements.

Findings

Training data quality is crucial for model performance.

Practitioners are largely unaware of TDA explanations.

End-users are open to TDA as a form of explanation.

Abstract

Explainable AI (XAI) aims to provide insight into opaque model reasoning to humans and as such is an interdisciplinary field by nature. In this paper, we interviewed 10 practitioners to understand the possible usability of training data attribution (TDA) explanations and to explore the design space of such an approach. We confirmed that training data quality is often the most important factor for high model performance in practice and model developers mainly rely on their own experience to curate data. End-users expect explanations to enhance their interaction with the model and do not necessarily prioritise but are open to training data as a means of explanation. Within our participants, we found that TDA explanations are not well-known and therefore not used. We urge the community to focus on the utility of TDA techniques from the human-machine collaboration perspective and broaden the TDA evaluation to reflect common use cases in practice.