FRWKV:Frequency-Domain Linear Attention for Long-Term Time Series Forecasting

TL;DR

FRWKV introduces a novel frequency-domain linear attention framework that combines spectral analysis with linear attention to enable scalable, efficient long-term time series forecasting, outperforming existing models.

Contribution

It presents a new model integrating linear attention with frequency-domain analysis, achieving linear complexity and superior performance in long-sequence time series forecasting.

Findings

FRWKV achieves first-place average rank across eight datasets.

Ablation studies highlight the importance of both linear attention and frequency encoding.

The model demonstrates scalable long-term forecasting with improved accuracy.

Abstract

Traditional Transformers face a major bottleneck in long-sequence time series forecasting due to their quadratic complexity $(\mathcal{O}(T^2))$ and their limited ability to effectively exploit frequency-domain information. Inspired by RWKV's $\mathcal{O}(T)$ linear attention and frequency-domain modeling, we propose FRWKV, a frequency-domain linear-attention framework that overcomes these limitations. Our model integrates linear attention mechanisms with frequency-domain analysis, achieving $\mathcal{O}(T)$ computational complexity in the attention path while exploiting spectral information to enhance temporal feature representations for scalable long-sequence modeling. Across eight real-world datasets, FRWKV achieves a first-place average rank. Our ablation studies confirm the critical roles of both the linear attention and frequency-encoder components. This work demonstrates the powerful synergy between linear attention and frequency analysis, establishing a new paradigm for scalable time series modeling. Code is available at this repository: https://github.com/yangqingyuan-byte/FRWKV.