OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain

TL;DR

OLinear introduces a novel orthogonal transformation-based linear model for multivariate time series forecasting, outperforming existing methods in accuracy and efficiency by decorrelating data and employing a specialized linear layer.

Contribution

The paper proposes a data-adaptive orthogonal transformation and a normalized linear layer, improving multivariate time series forecasting and serving as an efficient plug-in for existing models.

Findings

Outperforms state-of-the-art models on 24 benchmarks

Achieves high accuracy with nearly half the FLOPs of attention-based models

Enhances Transformer forecasters when used as a plug-in

Abstract

This paper presents $\mathbf{OLinear}$, a $\mathbf{linear}$-based multivariate time series forecasting model that operates in an $\mathbf{o}$rthogonally transformed domain. Recent forecasting models typically adopt the temporal forecast (TF) paradigm, which directly encode and decode time series in the time domain. However, the entangled step-wise dependencies in series data can hinder the performance of TF. To address this, some forecasters conduct encoding and decoding in the transformed domain using fixed, dataset-independent bases (e.g., sine and cosine signals in the Fourier transform). In contrast, we utilize $\mathbf{OrthoTrans}$, a data-adaptive transformation based on an orthogonal matrix that diagonalizes the series' temporal Pearson correlation matrix. This approach enables more effective encoding and decoding in the decorrelated feature domain and can serve as a plug-in module to enhance existing forecasters. To enhance the representation learning for multivariate time series, we introduce a customized linear layer, $\mathbf{NormLin}$, which employs a normalized weight matrix to capture multivariate dependencies. Empirically, the NormLin module shows a surprising performance advantage over multi-head self-attention, while requiring nearly half the FLOPs. Extensive experiments on 24 benchmarks and 140 forecasting tasks demonstrate that OLinear consistently achieves state-of-the-art performance with high efficiency. Notably, as a plug-in replacement for self-attention, the NormLin module consistently enhances Transformer-based forecasters. The code and datasets are available at https://anonymous.4open.science/r/OLinear