Leveraging Multivariate Long-Term History Representation for Time Series Forecasting

TL;DR

This paper introduces LMHR, a novel framework that enhances multivariate time series forecasting by capturing long-term spatial-temporal dependencies using specialized encoding, retrieval, and aggregation techniques, leading to improved accuracy.

Contribution

The paper proposes a new framework combining long-term history encoding, hierarchical retrieval, and transformer-based aggregation to better model long-term dependencies in multivariate time series forecasting.

Findings

LMHR outperforms typical STGNNs by 10.72% on average prediction horizons.

Achieves 4.12% improvement over state-of-the-art methods on multiple datasets.

Enhances accuracy by 9.8% on rapidly changing patterns.

Abstract

Multivariate Time Series (MTS) forecasting has a wide range of applications in both industry and academia. Recent advances in Spatial-Temporal Graph Neural Network (STGNN) have achieved great progress in modelling spatial-temporal correlations. Limited by computational complexity, most STGNNs for MTS forecasting focus primarily on short-term and local spatial-temporal dependencies. Although some recent methods attempt to incorporate univariate history into modeling, they still overlook crucial long-term spatial-temporal similarities and correlations across MTS, which are essential for accurate forecasting. To fill this gap, we propose a framework called the Long-term Multivariate History Representation (LMHR) Enhanced STGNN for MTS forecasting. Specifically, a Long-term History Encoder (LHEncoder) is adopted to effectively encode the long-term history into segment-level contextual representations and reduce point-level noise. A non-parametric Hierarchical Representation Retriever (HRetriever) is designed to include the spatial information in the long-term spatial-temporal dependency modelling and pick out the most valuable representations with no additional training. A Transformer-based Aggregator (TAggregator) selectively fuses the sparsely retrieved contextual representations based on the ranking positional embedding efficiently. Experimental results demonstrate that LMHR outperforms typical STGNNs by 10.72% on the average prediction horizons and state-of-the-art methods by 4.12% on several real-world datasets. Additionally, it consistently improves prediction accuracy by 9.8% on the top 10% of rapidly changing patterns across the datasets.