VE: Modeling Multivariate Time Series Correlation with Variate Embedding

TL;DR

This paper introduces the variate embedding (VE) pipeline, which learns unique embeddings for each variate in multivariate time series forecasting, improving correlation modeling and forecasting accuracy by integrating with existing models.

Contribution

The paper proposes a novel VE pipeline that captures variate correlations and can be integrated into any CI-based model, enhancing multivariate forecasting performance.

Findings

VE effectively groups similar variates and separates low-correlation ones.

Experiments on four datasets show improved forecasting accuracy.

The method controls parameter size using MoE and LoRA techniques.

Abstract

Multivariate time series forecasting relies on accurately capturing the correlations among variates. Current channel-independent (CI) models and models with a CI final projection layer are unable to capture these dependencies. In this paper, we present the variate embedding (VE) pipeline, which learns a unique and consistent embedding for each variate and combines it with Mixture of Experts (MoE) and Low-Rank Adaptation (LoRA) techniques to enhance forecasting performance while controlling parameter size. The VE pipeline can be integrated into any model with a CI final projection layer to improve multivariate forecasting. The learned VE effectively groups variates with similar temporal patterns and separates those with low correlations. The effectiveness of the VE pipeline is demonstrated through experiments on four widely-used datasets. The code is available at: https://github.com/swang-song/VE.