Less is more: Embracing sparsity and interpolation with Esiformer for time series forecasting

TL;DR

The paper introduces Esiformer, a novel time series forecasting model that uses data interpolation and a robust Sparse Feedforward Network to improve accuracy and robustness against noise, outperforming existing models.

Contribution

Esiformer combines data interpolation with a robust Sparse FFN to enhance forecasting accuracy and robustness, addressing noise and variance issues in real-world time series data.

Findings

Outperforms PatchTST with 6.5% lower MSE

Achieves 5.8% lower MAE on real-world datasets

Effectively reduces data variance and noise influence

Abstract

Time series forecasting has played a significant role in many practical fields. But time series data generated from real-world applications always exhibits high variance and lots of noise, which makes it difficult to capture the inherent periodic patterns of the data, hurting the prediction accuracy significantly. To address this issue, we propose the Esiformer, which apply interpolation on the original data, decreasing the overall variance of the data and alleviating the influence of noise. What's more, we enhanced the vanilla transformer with a robust Sparse FFN. It can enhance the representation ability of the model effectively, and maintain the excellent robustness, avoiding the risk of overfitting compared with the vanilla implementation. Through evaluations on challenging real-world datasets, our method outperforms leading model PatchTST, reducing MSE by 6.5% and MAE by 5.8% in multivariate time series forecasting. Code is available at: https://github.com/yyg1282142265/Esiformer/tree/main.