Variable Importance in Generalized Linear Models -- A Unifying View Using Shapley Values

TL;DR

This paper unifies the concept of variable importance in generalized linear models using Shapley values and introduces a Kullback-Leibler based pseudo-R^2 for consistent importance assessment.

Contribution

It proposes a unifying framework for variable importance in GLMs using Shapley values and introduces a new pseudo-R^2 measure based on Kullback-Leibler divergence.

Findings

Kullback-Leibler R^2 enables consistent importance measurement in GLMs.

The framework applies to both linear and nonlinear models.

Examples demonstrate practical utility in health and insurance data.

Abstract

Variable importance in regression analyses is of considerable interest in a variety of fields. There is no unique method for assessing variable importance. However, a substantial share of the available literature employs Shapley values, either explicitly or implicitly, to decompose a suitable goodness-of-fit measure, in the linear regression model typically the classical $R^2$. Beyond linear regression, there is no generally accepted goodness-of-fit measure, only a variety of pseudo-$R^2$s. We formulate and discuss the desirable properties of goodness-of-fit measures that enable Shapley values to be interpreted in terms of relative, and even absolute, importance. We suggest to use a pseudo-$R^2$ based on the Kullback-Leibler divergence, the Kullback-Leibler $R^2$, which has a convenient form for generalized linear models and permits to unify and extend previous work on variable importance for linear and nonlinear models. Several examples are presented, using data from public health and insurance.