InstaSHAP: Interpretable Additive Models Explain Shapley Values Instantly

TL;DR

This paper introduces InstaSHAP, a method that leverages additive models to efficiently compute Shapley values for model explanations, revealing limitations in GAM models and their impact on SHAP explanations.

Contribution

It establishes a variational link between GAM models and SHAP, enabling fast Shapley value computation and analyzing the representation limits of GAMs in explanation methods.

Findings

GAM models can be trained to compute Shapley values in a single forward pass.

The limited representation power of GAMs affects the effectiveness of SHAP explanations.

Theoretical analysis shows GAMs' constraints mirror those in SHAP, impacting CV and NLP applications.

Abstract

In recent years, the Shapley value and SHAP explanations have emerged as one of the most dominant paradigms for providing post-hoc explanations of black-box models. Despite their well-founded theoretical properties, many recent works have focused on the limitations in both their computational efficiency and their representation power. The underlying connection with additive models, however, is left critically under-emphasized in the current literature. In this work, we find that a variational perspective linking GAM models and SHAP explanations is able to provide deep insights into nearly all recent developments. In light of this connection, we borrow in the other direction to develop a new method to train interpretable GAM models which are automatically purified to compute the Shapley value in a single forward pass. Finally, we provide theoretical results showing the limited representation power of GAM models is the same Achilles' heel existing in SHAP and discuss the implications for SHAP's modern usage in CV and NLP.