A principled approach for comparing Variable Importance

TL;DR

This paper introduces an axiomatic framework and a systematic pipeline for evaluating and comparing variable importance measures, aiming to improve their interpretability and reliability in explainable AI.

Contribution

It proposes a formal axiomatic approach linking variable importance to variable selection and offers a general pipeline for constructing and comparing VIMs.

Findings

Framework avoids false positives from spurious correlations

Pipeline clarifies objectives of different VIMs

Guidelines for practitioners to select suitable VIMs

Abstract

Variable importance measures (VIMs) aim to quantify the contribution of each input covariate to the predictability of a given output. With the growing interest in explainable AI, numerous VIMs have been proposed, many of which are heuristic in nature. This is often justified by the inherent subjectivity of the notion of importance. This raises important questions regarding usage: What makes a good VIM? How can we compare different VIMs? In this paper, we address these questions by: (1) proposing an axiomatic framework that bridges the gap between variable importance and variable selection. This framework formalizes the intuitive principle that features providing no additional information should not be assigned importance. It helps avoid false positives due to spurious correlations, which can arise with popular methods such as Shapley values; and (2) introducing a general pipeline for constructing VIMs, which clarifies the objective of various VIMs and thus facilitates meaningful comparisons. This approach is natural in statistics, but the literature has diverged from it. Finally, we provide an extensive set of examples to guide practitioners in selecting and estimating appropriate indices aligned with their specific goals and data.