Distributing Synergy Functions: Unifying Game-Theoretic Interaction Methods for Machine-Learning Explainability

TL;DR

This paper introduces a unifying framework for game-theoretic attribution and interaction methods in machine learning, defining a unique way to distribute feature synergies and analyzing gradient-based approaches for model explainability.

Contribution

It presents a comprehensive framework that unifies various attribution and interaction methods, characterizes gradient-based approaches, and highlights the importance of goal-oriented method development.

Findings

Unique full account of feature interactions (synergies) in continuous input settings.

Gradient-based methods are characterized by their actions on monomials.

Combination of criteria defines attribution and interaction methods uniquely.

Abstract

Deep learning has revolutionized many areas of machine learning, from computer vision to natural language processing, but these high-performance models are generally "black box." Explaining such models would improve transparency and trust in AI-powered decision making and is necessary for understanding other practical needs such as robustness and fairness. A popular means of enhancing model transparency is to quantify how individual inputs contribute to model outputs (called attributions) and the magnitude of interactions between groups of inputs. A growing number of these methods import concepts and results from game theory to produce attributions and interactions. This work presents a unifying framework for game-theory-inspired attribution and $k^\text{th}$-order interaction methods. We show that, given modest assumptions, a unique full account of interactions between features, called synergies, is possible in the continuous input setting. We identify how various methods are characterized by their policy of distributing synergies. We also demonstrate that gradient-based methods are characterized by their actions on monomials, a type of synergy function, and introduce unique gradient-based methods. We show that the combination of various criteria uniquely defines the attribution/interaction methods. Thus, the community needs to identify goals and contexts when developing and employing attribution and interaction methods.