On the influence of dependent features in classification problems: a game-theoretic perspective

TL;DR

This paper introduces a new influence measure for features in classification tasks that accounts for dependencies among features, using a game-theoretic approach, and relates it to the Banzhaf-Owen value.

Contribution

It proposes a novel influence measure for dependent features in classification, characterized axiomatically and linked to cooperative game theory, specifically the Banzhaf-Owen value.

Findings

The influence measure generalizes existing importance metrics.

It is characterized by axioms adapted from cooperative game theory.

Numerical examples demonstrate practical applications.

Abstract

This paper deals with a new measure of the influence of each feature on the response variable in classification problems, accounting for potential dependencies among certain feature subsets. Within this framework, we consider a sample of individuals characterized by specific features, each feature encompassing a finite range of values, and classified based on a binary response variable. This measure turns out to be an influence measure explored in existing literature and related to cooperative game theory. We provide an axiomatic characterization of our proposed influence measure by tailoring properties from the cooperative game theory to our specific context. Furthermore, we demonstrate that our influence measure becomes a general characterization of the well-known Banzhaf-Owen value for games with a priori unions, from the perspective of classification problems. The definitions and results presented herein are illustrated through numerical examples and various applications, offering practical insights into our methodologies.