Unifying Prediction and Explanation in Time-Series Transformers via Shapley-based Pretraining

TL;DR

ShapTST introduces a pre-training framework for time-series transformers that unifies prediction and explanation, significantly reducing explanation computation costs while improving robustness and maintaining competitive accuracy.

Contribution

The paper presents a novel Shapley-based pre-training method that integrates explanation generation into training, eliminating expensive inference-time computations for explanations.

Findings

Achieved competitive classification and regression results on eight datasets.

Provided Shapley-value explanations comparable to post-hoc methods.

Enhanced model robustness against data noise.

Abstract

In this paper, we propose ShapTST, a framework that enables time-series transformers to efficiently generate Shapley-value-based explanations alongside predictions in a single forward pass. Shapley values are widely used to evaluate the contribution of different time-steps and features in a test sample, and are commonly generated through repeatedly inferring on each sample with different parts of information removed. Therefore, it requires expensive inference-time computations that occur at every request for model explanations. In contrast, our framework unifies the explanation and prediction in training through a novel Shapley-based pre-training design, which eliminates the undesirable test-time computation and replaces it with a single-time pre-training. Moreover, this specialized pre-training benefits the prediction performance by making the transformer model more effectively weigh different features and time-steps in the time-series, particularly improving the robustness against data noise that is common to raw time-series data. We experimentally validated our approach on eight public datasets, where our time-series model achieved competitive results in both classification and regression tasks, while providing Shapley-based explanations similar to those obtained with post-hoc computation. Our work offers an efficient and explainable solution for time-series analysis tasks in the safety-critical applications.