Feature Importance Explanations for Temporal Black-Box Models

TL;DR

This paper introduces TIME, a model-agnostic method for explaining the importance of features in temporal black-box models, incorporating temporal ordering, localized influence, and statistical testing for interpretability.

Contribution

The paper presents TIME, a novel explanation method specifically designed for temporal models, addressing the limitations of existing approaches that lack temporal considerations.

Findings

TIME effectively captures temporal feature importance.

The method provides statistically rigorous explanations.

It identifies salient features over time and localized windows.

Abstract

Models in the supervised learning framework may capture rich and complex representations over the features that are hard for humans to interpret. Existing methods to explain such models are often specific to architectures and data where the features do not have a time-varying component. In this work, we propose TIME, a method to explain models that are inherently temporal in nature. Our approach (i) uses a model-agnostic permutation-based approach to analyze global feature importance, (ii) identifies the importance of salient features with respect to their temporal ordering as well as localized windows of influence, and (iii) uses hypothesis testing to provide statistical rigor.