The Berkelmans-Pries Feature Importance Method: A Generic Measure of Informativeness of Features

TL;DR

The paper introduces the Berkelmans-Pries Feature Importance method, a new global measure combining Shapley values and dependency functions, which accurately estimates feature importance in datasets with complex interdependencies.

Contribution

A novel FI method based on Shapley values and Berkelmans-Pries dependency, with proven properties and superior accuracy over existing methods.

Findings

Accurately predicts ground truth FI in test cases

Outperforms 468 existing FI methods in property analysis

Effective for datasets with complex feature interdependencies

Abstract

Over the past few years, the use of machine learning models has emerged as a generic and powerful means for prediction purposes. At the same time, there is a growing demand for interpretability of prediction models. To determine which features of a dataset are important to predict a target variable $Y$, a Feature Importance (FI) method can be used. By quantifying how important each feature is for predicting $Y$, irrelevant features can be identified and removed, which could increase the speed and accuracy of a model, and moreover, important features can be discovered, which could lead to valuable insights. A major problem with evaluating FI methods, is that the ground truth FI is often unknown. As a consequence, existing FI methods do not give the exact correct FI values. This is one of the many reasons why it can be hard to properly interpret the results of an FI method. Motivated by this, we introduce a new global approach named the Berkelmans-Pries FI method, which is based on a combination of Shapley values and the Berkelmans-Pries dependency function. We prove that our method has many useful properties, and accurately predicts the correct FI values for several cases where the ground truth FI can be derived in an exact manner. We experimentally show for a large collection of FI methods (468) that existing methods do not have the same useful properties. This shows that the Berkelmans-Pries FI method is a highly valuable tool for analyzing datasets with complex interdependencies.