FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecasting

TL;DR

FECAM introduces a frequency enhanced channel attention mechanism using Discrete Cosine Transform to improve time series forecasting accuracy across various models by effectively capturing frequency information and avoiding Fourier Transform issues.

Contribution

The paper proposes a novel frequency enhanced channel attention module based on DCT, which can be integrated into existing models to improve forecasting performance.

Findings

Achieves state-of-the-art results on six real-world datasets.

Reduces MSE by up to 36% on LSTM and improves other models.

Can be applied flexibly with minimal computational overhead.

Abstract

Time series forecasting is a long-standing challenge due to the real-world information is in various scenario (e.g., energy, weather, traffic, economics, earthquake warning). However some mainstream forecasting model forecasting result is derailed dramatically from ground truth. We believe it's the reason that model's lacking ability of capturing frequency information which richly contains in real world datasets. At present, the mainstream frequency information extraction methods are Fourier transform(FT) based. However, use of FT is problematic due to Gibbs phenomenon. If the values on both sides of sequences differ significantly, oscillatory approximations are observed around both sides and high frequency noise will be introduced. Therefore We propose a novel frequency enhanced channel attention that adaptively modelling frequency interdependencies between channels based on Discrete Cosine Transform which would intrinsically avoid high frequency noise caused by problematic periodity during Fourier Transform, which is defined as Gibbs Phenomenon. We show that this network generalize extremely effectively across six real-world datasets and achieve state-of-the-art performance, we further demonstrate that frequency enhanced channel attention mechanism module can be flexibly applied to different networks. This module can improve the prediction ability of existing mainstream networks, which reduces 35.99% MSE on LSTM, 10.01% on Reformer, 8.71% on Informer, 8.29% on Autoformer, 8.06% on Transformer, etc., at a slight computational cost ,with just a few line of code. Our codes and data are available at https://github.com/Zero-coder/FECAM.