Towards Understanding the Influence of Training Samples on Explanations

TL;DR

This paper explores how individual training samples influence explanations in AI models, introducing a new method to identify influential data points affecting model reasoning and recourse costs.

Contribution

It presents a novel approach to determine training samples that shape model explanations, extending data valuation to interpretability and recourse analysis.

Findings

Identified training samples that influence explanations and recourse costs.

Demonstrated the effectiveness of the proposed algorithm in two case studies.

Provided insights into data's role in model interpretability and decision-making.

Abstract

Explainable AI (XAI) is widely used to analyze AI systems' decision-making, such as providing counterfactual explanations for recourse. When unexpected explanations occur, users may want to understand the training data properties shaping them. Under the umbrella of data valuation, first approaches have been proposed that estimate the influence of data samples on a given model. This process not only helps determine the data's value, but also offers insights into how individual, potentially noisy, or misleading examples affect a model, which is crucial for interpretable AI. In this work, we apply the concept of data valuation to the significant area of model evaluations, focusing on how individual training samples impact a model's internal reasoning rather than the predictive performance only. Hence, we introduce the novel problem of identifying training samples shaping a given explanation or related quantity, and investigate the particular case of the cost of computational recourse. We propose an algorithm to identify such influential samples and conduct extensive empirical evaluations in two case studies.