Provably Adaptive Linear Approximation for the Shapley Value and Beyond

TL;DR

This paper introduces Adalina, an adaptive, linear-space algorithm for efficiently approximating semi-values like the Shapley value with provable accuracy guarantees, suitable for large-scale applications.

Contribution

It develops a theoretical framework for sharper query complexity bounds and presents the first adaptive, linear-time, linear-space algorithm for semi-value approximation.

Findings

The framework enables sharper query complexity bounds for existing algorithms.

Adalina achieves improved mean square error in approximations.

Experimental results validate the theoretical improvements.

Abstract

The Shapley value, and its broader family of semi-values, has received much attention in various attribution problems. A fundamental and long-standing challenge is their efficient approximation, since exact computation generally requires an exponential number of utility queries in the number of players $n$. To meet the challenges of large-scale applications, we explore the limits of efficiently approximating semi-values under a $\Theta(n)$ space constraint. Building upon a vector concentration inequality, we establish a theoretical framework that enables sharper query complexities for existing unbiased randomized algorithms. Within this framework, we systematically develop a linear-space algorithm that requires $O(\frac{n}{\epsilon^{2}}\log\frac{1}{\delta})$ utility queries to ensure $P(\|\hat{\boldsymbol\phi}-\boldsymbol\phi\|_{2}\geq\epsilon)\leq \delta$ for all commonly used semi-values. In particular, our framework naturally bridges OFA, unbiased kernelSHAP, SHAP-IQ and the regression-adjusted approach, and definitively characterizes when paired sampling is beneficial. Moreover, our algorithm allows explicit minimization of the mean square error for each specific utility function. Accordingly, we introduce the first adaptive, linear-time, linear-space randomized algorithm, Adalina, that theoretically achieves improved mean square error. All of our theoretical findings are experimentally validated.