Ister: Linear Transformer for Efficient Multivariate Time Series Forecasting

TL;DR

Ister is a novel linear-transformer architecture for multivariate time series forecasting that improves efficiency and accuracy by using a linear attention mechanism and seasonal-trend decomposition.

Contribution

The paper introduces Ister, combining a linear attention mechanism with seasonal-trend decomposition for scalable and accurate multivariate time series forecasting.

Findings

Ister achieves state-of-the-art results on multiple benchmarks.

The linear attention mechanism reduces computational complexity.

Seasonal-trend decomposition enhances channel alignment.

Abstract

Transformer-based models have achieved remarkable success in multivariate time series forecasting (MTSF) by capturing long-range dependencies. However, their widespread adoption is hindered by the quadratic computational complexity of self-attention, which limits scalability on high-dimensional sequences. To address this challenge, we propose the Inverted Seasonal-Trend Decomposition Transformer (Ister), a novel architecture that enhances both predictive accuracy and computational efficiency. Central to Ister is Dot-attention, a linear-complexity attention mechanism that replaces conventional multi-head self-attention with element-wise dot-product operations to model inter-series dependencies. Furthermore, we introduce an inverted seasonal-trend decomposition strategy that isolates periodic components, enabling the model to focus learning on periodic patterns, thereby improving the performance of channel alignment. Extensive experiments across several real-world benchmarks demonstrate that Ister consistently achieves state-of-the-art performance. Code is available at https://github.com/macovaseas/Ister.