Evaluating Explanation Methods for Multivariate Time Series Classification

TL;DR

This paper evaluates explanation methods for multivariate time series classification, analyzing their effectiveness on synthetic and real datasets, and finds that simple concatenation and adapted SHAP methods perform well.

Contribution

It provides a comprehensive analysis of saliency-based explanation methods for multivariate time series classifiers, highlighting effective approaches and dataset limitations.

Findings

Concatenating channels performs as well as multivariate classifiers.

Adapted SHAP methods work effectively for MTSC.

Synthetic datasets may not be suitable for time series analysis.

Abstract

Multivariate time series classification is an important computational task arising in applications where data is recorded over time and over multiple channels. For example, a smartwatch can record the acceleration and orientation of a person's motion, and these signals are recorded as multivariate time series. We can classify this data to understand and predict human movement and various properties such as fitness levels. In many applications classification alone is not enough, we often need to classify but also understand what the model learns (e.g., why was a prediction given, based on what information in the data). The main focus of this paper is on analysing and evaluating explanation methods tailored to Multivariate Time Series Classification (MTSC). We focus on saliency-based explanation methods that can point out the most relevant channels and time series points for the classification decision. We analyse two popular and accurate multivariate time series classifiers, ROCKET and dResNet, as well as two popular explanation methods, SHAP and dCAM. We study these methods on 3 synthetic datasets and 2 real-world datasets and provide a quantitative and qualitative analysis of the explanations provided. We find that flattening the multivariate datasets by concatenating the channels works as well as using multivariate classifiers directly and adaptations of SHAP for MTSC work quite well. Additionally, we also find that the popular synthetic datasets we used are not suitable for time series analysis.