Game-Theoretic Interpretability for Temporal Modeling

TL;DR

This paper introduces a game-theoretic framework for interpreting temporal sequence models by co-operatively analyzing how well predictors conform to interpretable temporal families, enhancing interpretability without restricting model classes.

Contribution

It proposes a novel cooperative game approach for interpretability in temporal modeling that does not impose prior restrictions on predictor classes.

Findings

Framework effectively highlights local conformity to interpretable models

Demonstrations on temporal sequence models validate the approach

Enhances interpretability without sacrificing model flexibility

Abstract

Interpretability has arisen as a key desideratum of machine learning models alongside performance. Approaches so far have been primarily concerned with fixed dimensional inputs emphasizing feature relevance or selection. In contrast, we focus on temporal modeling and the problem of tailoring the predictor, functionally, towards an interpretable family. To this end, we propose a co-operative game between the predictor and an explainer without any a priori restrictions on the functional class of the predictor. The goal of the explainer is to highlight, locally, how well the predictor conforms to the chosen interpretable family of temporal models. Our co-operative game is setup asymmetrically in terms of information sets for efficiency reasons. We develop and illustrate the framework in the context of temporal sequence models with examples.