PolySHAP: Extending KernelSHAP with Interaction-Informed Polynomial Regression

TL;DR

PolySHAP extends KernelSHAP by incorporating higher degree polynomials to better capture feature interactions, resulting in more accurate Shapley value estimates with theoretical guarantees.

Contribution

It introduces PolySHAP, a polynomial-based extension of KernelSHAP, and establishes a theoretical link between paired sampling and second-order polynomial approximation.

Findings

PolySHAP empirically improves Shapley value estimates on benchmark datasets.

Theoretical proof shows paired sampling matches second-order PolySHAP without polynomial fitting.

PolySHAP provides consistent estimates and justifies paired sampling's effectiveness.

Abstract

Shapley values have emerged as a central game-theoretic tool in explainable AI (XAI). However, computing Shapley values exactly requires $2^d$ game evaluations for a model with $d$ features. Lundberg and Lee's KernelSHAP algorithm has emerged as a leading method for avoiding this exponential cost. KernelSHAP approximates Shapley values by approximating the game as a linear function, which is fit using a small number of game evaluations for random feature subsets. In this work, we extend KernelSHAP by approximating the game via higher degree polynomials, which capture non-linear interactions between features. Our resulting PolySHAP method yields empirically better Shapley value estimates for various benchmark datasets, and we prove that these estimates are consistent. Moreover, we connect our approach to paired sampling (antithetic sampling), a ubiquitous modification to KernelSHAP that improves empirical accuracy. We prove that paired sampling outputs exactly the same Shapley value approximations as second-order PolySHAP, without ever fitting a degree 2 polynomial. To the best of our knowledge, this finding provides the first strong theoretical justification for the excellent practical performance of the paired sampling heuristic.