Parallel Spatio-Temporal Attention-Based TCN for Multivariate Time Series Prediction

TL;DR

This paper introduces PSTA-TCN, a novel framework combining parallel spatio-temporal attention with stacked TCNs, significantly improving long-horizon multivariate time series prediction in terms of accuracy and training efficiency.

Contribution

The paper presents a new framework that integrates parallel spatio-temporal attention with TCNs, addressing limitations of RNNs for long-term forecasting in complex multivariate data.

Findings

Dramatic reduction in training times due to parallel computation

Substantial increase in prediction accuracy for long horizons

Prediction windows up to 13 times longer than existing methods

Abstract

As industrial systems become more complex and monitoring sensors for everything from surveillance to our health become more ubiquitous, multivariate time series prediction is taking an important place in the smooth-running of our society. A recurrent neural network with attention to help extend the prediction windows is the current-state-of-the-art for this task. However, we argue that their vanishing gradients, short memories, and serial architecture make RNNs fundamentally unsuited to long-horizon forecasting with complex data. Temporal convolutional networks (TCNs) do not suffer from gradient problems and they support parallel calculations, making them a more appropriate choice. Additionally, they have longer memories than RNNs, albeit with some instability and efficiency problems. Hence, we propose a framework, called PSTA-TCN, that combines a parallel spatio-temporal attention mechanism to extract dynamic internal correlations with stacked TCN backbones to extract features from different window sizes. The framework makes full use parallel calculations to dramatically reduce training times, while substantially increasing accuracy with stable prediction windows up to 13 times longer than the status quo.