Shapley Marginal Surplus for Strong Models

TL;DR

This paper introduces a new method called Shapley Marginal Surplus for Strong Models that improves the inference of feature importance in complex models, especially when variables are interrelated or noisy.

Contribution

The paper proposes a novel variable importance algorithm that samples model space to better infer true feature importance, outperforming existing methods.

Findings

The new method outperforms other feature importance techniques in inferential accuracy.

Model explanations may not reflect the true data-generating process, especially with noisy or correlated variables.

Sampling model space enhances the interpretability of feature importance in complex models.

Abstract

Shapley values have seen widespread use in machine learning as a way to explain model predictions and estimate the importance of covariates. Accurately explaining models is critical in real-world models to both aid in decision making and to infer the properties of the true data-generating process (DGP). In this paper, we demonstrate that while model-based Shapley values might be accurate explainers of model predictions, machine learning models themselves are often poor explainers of the DGP even if the model is highly accurate. Particularly in the presence of interrelated or noisy variables, the output of a highly predictive model may fail to account for these relationships. This implies explanations of a trained model's behavior may fail to provide meaningful insight into the DGP. In this paper we introduce a novel variable importance algorithm, Shapley Marginal Surplus for Strong Models, that samples the space of possible models to come up with an inferential measure of feature importance. We compare this method to other popular feature importance methods, both Shapley-based and non-Shapley based, and demonstrate significant outperformance in inferential capabilities relative to other methods.