SEER: Transformer-based Robust Time Series Forecasting via Automated Patch Enhancement and Replacement

TL;DR

SEER is a transformer-based framework that enhances time series forecasting robustness by dynamically selecting and replacing low-quality patches, leading to state-of-the-art results in handling real-world data issues.

Contribution

The paper introduces SEER, a novel robust forecasting framework with an augmented embedding module and a learnable patch replacement mechanism for improved accuracy and resilience.

Findings

Achieves state-of-the-art forecasting accuracy on multiple datasets.

Effectively handles missing data, anomalies, and noise in real-world time series.

Demonstrates robustness against low-quality data issues.

Abstract

Time series forecasting is important in many fields that require accurate predictions for decision-making. Patching techniques, commonly used and effective in time series modeling, help capture temporal dependencies by dividing the data into patches. However, existing patch-based methods fail to dynamically select patches and typically use all patches during the prediction process. In real-world time series, there are often low-quality issues during data collection, such as missing values, distribution shifts, anomalies and white noise, which may cause some patches to contain low-quality information, negatively impacting the prediction results. To address this issue, this study proposes a robust time series forecasting framework called SEER. Firstly, we propose an Augmented Embedding Module, which improves patch-wise representations using a Mixture-of-Experts (MoE) architecture and obtains series-wise token representations through a channel-adaptive perception mechanism. Secondly, we introduce a Learnable Patch Replacement Module, which enhances forecasting robustness and model accuracy through a two-stage process: 1) a dynamic filtering mechanism eliminates negative patch-wise tokens; 2) a replaced attention module substitutes the identified low-quality patches with global series-wise token, further refining their representations through a causal attention mechanism. Comprehensive experimental results demonstrate the SOTA performance of SEER.