Don't Get Me Wrong: How to Apply Deep Visual Interpretations to Time Series

TL;DR

This paper evaluates various saliency methods for interpreting convolutional models on complex time series data, providing guidelines for selecting appropriate methods based on comprehensive assessments.

Contribution

It offers a systematic evaluation of nine saliency methods on diverse time series datasets and formulates practical recommendations for their application.

Findings

No single saliency method outperforms others across all metrics

Some methods perform better on specific datasets or metrics

Guidelines help experts choose suitable interpretability techniques

Abstract

The correct interpretation of convolutional models is a hard problem for time series data. While saliency methods promise visual validation of predictions for image and language processing, they fall short when applied to time series. These tend to be less intuitive and represent highly diverse data, such as the tool-use time series dataset. Furthermore, saliency methods often generate varied, conflicting explanations, complicating the reliability of these methods. Consequently, a rigorous objective assessment is necessary to establish trust in them. This paper investigates saliency methods on time series data to formulate recommendations for interpreting convolutional models and implements them on the tool-use time series problem. To achieve this, we first employ nine gradient-, propagation-, or perturbation-based post-hoc saliency methods across six varied and complex real-world datasets. Next, we evaluate these methods using five independent metrics to generate recommendations. Subsequently, we implement a case study focusing on tool-use time series using convolutional classification models. Our results validate our recommendations that indicate that none of the saliency methods consistently outperforms others on all metrics, while some are sometimes ahead. Our insights and step-by-step guidelines allow experts to choose suitable saliency methods for a given model and dataset.