Statistical Inference and Learning for Shapley Additive Explanations (SHAP)

TL;DR

This paper develops statistical inference methods for global feature importance measures derived from SHAP explanations, enabling reliable confidence intervals and hypothesis testing for these importance scores.

Contribution

It introduces semi-parametric approaches for constructing asymptotically normal estimates and de-biased U-statistics for SHAP-based importance measures, addressing a gap in statistical inference.

Findings

Asymptotically normal estimates for SHAP importance measures are derived.

De-biased U-statistics are proposed for p ≥ 2 and smoothed alternatives for 1 ≤ p < 2.

A Neyman orthogonal loss function is developed for learning SHAP curves with risk guarantees.

Abstract

The SHAP (short for Shapley additive explanation) framework has become an essential tool for attributing importance to variables in predictive tasks. In model-agnostic settings, SHAP uses the concept of Shapley values from cooperative game theory to fairly allocate credit to the features in a vector $X$ based on their contribution to an outcome $Y$. While the explanations offered by SHAP are local by nature, learners often need global measures of feature importance in order to improve model explainability and perform feature selection. The most common approach for converting these local explanations into global ones is to compute either the mean absolute SHAP or mean squared SHAP. However, despite their ubiquity, there do not exist approaches for performing statistical inference on these quantities. In this paper, we take a semi-parametric approach for calibrating confidence in estimates of the $p$th powers of Shapley additive explanations. We show that, by treating the SHAP curve as a nuisance function that must be estimated from data, one can reliably construct asymptotically normal estimates of the $p$th powers of SHAP. When $p \geq 2$, we show a de-biased estimator that combines U-statistics with Neyman orthogonal scores for functionals of nested regressions is asymptotically normal. When $1 \leq p < 2$ (and the hence target parameter is not twice differentiable), we construct de-biased U-statistics for a smoothed alternative. In particular, we show how to carefully tune the temperature parameter of the smoothing function in order to obtain inference for the true, unsmoothed $p$th power. We complement these results by presenting a Neyman orthogonal loss that can be used to learn the SHAP curve via empirical risk minimization and discussing excess risk guarantees for commonly used function classes.