ACFormer: Mitigating Non-linearity with Auto Convolutional Encoder for Time Series Forecasting

TL;DR

ACFormer is a novel architecture that combines linear efficiency with convolutional non-linearity to improve time series forecasting, especially for complex, high-frequency signals.

Contribution

The paper introduces ACFormer, a new model that integrates insights from receptive field analysis to better capture non-linear temporal dependencies in time series data.

Findings

ACFormer outperforms existing models on multiple benchmarks.

It effectively captures high-frequency and non-linear components.

The model demonstrates robustness to non-linear fluctuations.

Abstract

Time series forecasting (TSF) faces challenges in modeling complex intra-channel temporal dependencies and inter-channel correlations. Although recent research has highlighted the efficiency of linear architectures in capturing global trends, these models often struggle with non-linear signals. To address this gap, we conducted a systematic receptive field analysis of convolutional neural network (CNN) TSF models. We introduce the "individual receptive field" to uncover granular structural dependencies, revealing that convolutional layers act as feature extractors that mirror channel-wise attention while exhibiting superior robustness to non-linear fluctuations. Based on these insights, we propose ACFormer, an architecture designed to reconcile the efficiency of linear projections with the non-linear feature-extraction power of convolutions. ACFormer captures fine-grained information through a shared compression module, preserves temporal locality via gated attention, and reconstructs variable-specific temporal patterns using an independent patch expansion layer. Extensive experiments on multiple benchmark datasets demonstrate that ACFormer consistently achieves state-of-the-art performance, effectively mitigating the inherent drawbacks of linear models in capturing high-frequency components.