Towards a Rigorous Evaluation of XAI Methods on Time Series

TL;DR

This paper evaluates the effectiveness of various XAI methods on time series data, introducing new verification techniques to account for temporal aspects and assessing their robustness and architecture-specific performance.

Contribution

It presents a novel methodology for evaluating XAI methods on time series, incorporating temporal verification techniques and conducting initial experiments on multiple datasets.

Findings

SHAP is consistently robust across models

DeepLIFT, LRP, and Saliency Maps perform better with specific architectures

New verification techniques effectively incorporate temporal dimensions

Abstract

Explainable Artificial Intelligence (XAI) methods are typically deployed to explain and debug black-box machine learning models. However, most proposed XAI methods are black-boxes themselves and designed for images. Thus, they rely on visual interpretability to evaluate and prove explanations. In this work, we apply XAI methods previously used in the image and text-domain on time series. We present a methodology to test and evaluate various XAI methods on time series by introducing new verification techniques to incorporate the temporal dimension. We further conduct preliminary experiments to assess the quality of selected XAI method explanations with various verification methods on a range of datasets and inspecting quality metrics on it. We demonstrate that in our initial experiments, SHAP works robust for all models, but others like DeepLIFT, LRP, and Saliency Maps work better with specific architectures.