TimeSQL: Improving Multivariate Time Series Forecasting with Multi-Scale Patching and Smooth Quadratic Loss

TL;DR

TimeSQL introduces a multi-scale patching approach and a smooth quadratic loss function to improve multivariate time series forecasting, effectively handling noise and complex temporal dynamics, and achieves state-of-the-art results on benchmark datasets.

Contribution

The paper presents a novel framework combining multi-scale patching and SQL loss, which enhances forecasting accuracy and robustness in multivariate time series prediction.

Findings

Achieves state-of-the-art performance on eight benchmark datasets.

Multi-scale patching captures both local and global temporal features.

SQL loss reduces overfitting to noise and outliers.

Abstract

Time series is a special type of sequence data, a sequence of real-valued random variables collected at even intervals of time. The real-world multivariate time series comes with noises and contains complicated local and global temporal dynamics, making it difficult to forecast the future time series given the historical observations. This work proposes a simple and effective framework, coined as TimeSQL, which leverages multi-scale patching and smooth quadratic loss (SQL) to tackle the above challenges. The multi-scale patching transforms the time series into two-dimensional patches with different length scales, facilitating the perception of both locality and long-term correlations in time series. SQL is derived from the rational quadratic kernel and can dynamically adjust the gradients to avoid overfitting to the noises and outliers. Theoretical analysis demonstrates that, under mild conditions, the effect of the noises on the model with SQL is always smaller than that with MSE. Based on the two modules, TimeSQL achieves new state-of-the-art performance on the eight real-world benchmark datasets. Further ablation studies indicate that the key modules in TimeSQL could also enhance the results of other models for multivariate time series forecasting, standing as plug-and-play techniques.