On Model Extrapolation in Marginal Shapley Values

TL;DR

This paper investigates the issues of model extrapolation in marginal Shapley values, demonstrating its effects, proposing a method to avoid it, and showing how causal information can improve explanations in complex models.

Contribution

It introduces a new approach that avoids model extrapolation in marginal Shapley values and incorporates causal information to replicate causal Shapley values.

Findings

Marginal Shapley values can lead to model extrapolation issues.

The proposed method avoids extrapolation while maintaining interpretability.

Incorporating causal information improves the accuracy of Shapley value explanations.

Abstract

As the use of complex machine learning models continues to grow, so does the need for reliable explainability methods. One of the most popular methods for model explainability is based on Shapley values. There are two most commonly used approaches to calculating Shapley values which produce different results when features are correlated, conditional and marginal. In our previous work, it was demonstrated that the conditional approach is fundamentally flawed due to implicit assumptions of causality. However, it is a well-known fact that marginal approach to calculating Shapley values leads to model extrapolation where it might not be well defined. In this paper we explore the impacts of model extrapolation on Shapley values in the case of a simple linear spline model. Furthermore, we propose an approach which while using marginal averaging avoids model extrapolation and with addition of causal information replicates causal Shapley values. Finally, we demonstrate our method on the real data example.