VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting

TL;DR

VCformer introduces a novel transformer model that effectively captures variable correlations and temporal dependencies in multivariate time series, outperforming existing methods in real-world forecasting tasks.

Contribution

The paper proposes VCformer with Variable Correlation Attention and Koopman Temporal Detector, advancing multivariate time series forecasting by modeling cross-variable correlations and non-stationarity.

Findings

Achieves top-tier performance on eight real-world datasets.

Effectively captures cross-variable correlations with Variable Correlation Attention.

Addresses non-stationarity using Koopman Temporal Detector.

Abstract

Multivariate time series (MTS) forecasting has been extensively applied across diverse domains, such as weather prediction and energy consumption. However, current studies still rely on the vanilla point-wise self-attention mechanism to capture cross-variable dependencies, which is inadequate in extracting the intricate cross-correlation implied between variables. To fill this gap, we propose Variable Correlation Transformer (VCformer), which utilizes Variable Correlation Attention (VCA) module to mine the correlations among variables. Specifically, based on the stochastic process theory, VCA calculates and integrates the cross-correlation scores corresponding to different lags between queries and keys, thereby enhancing its ability to uncover multivariate relationships. Additionally, inspired by Koopman dynamics theory, we also develop Koopman Temporal Detector (KTD) to better address the non-stationarity in time series. The two key components enable VCformer to extract both multivariate correlations and temporal dependencies. Our extensive experiments on eight real-world datasets demonstrate the effectiveness of VCformer, achieving top-tier performance compared to other state-of-the-art baseline models. Code is available at this repository: https://github.com/CSyyn/VCformer.