MultiResFormer: Transformer with Adaptive Multi-Resolution Modeling for General Time Series Forecasting

TL;DR

MultiResFormer introduces an adaptive multi-resolution transformer model that dynamically selects optimal patch lengths to better capture complex temporal dependencies in time series forecasting, outperforming existing methods.

Contribution

It proposes a novel transformer architecture that adaptively models temporal variations with multiple patch lengths, enhancing forecasting accuracy.

Findings

Outperforms state-of-the-art patch-based transformers in long-term forecasting.

Consistently outperforms CNN baselines by a large margin.

Uses fewer parameters than comparable models.

Abstract

Transformer-based models have greatly pushed the boundaries of time series forecasting recently. Existing methods typically encode time series data into $\textit{patches}$ using one or a fixed set of patch lengths. This, however, could result in a lack of ability to capture the variety of intricate temporal dependencies present in real-world multi-periodic time series. In this paper, we propose MultiResFormer, which dynamically models temporal variations by adaptively choosing optimal patch lengths. Concretely, at the beginning of each layer, time series data is encoded into several parallel branches, each using a detected periodicity, before going through the transformer encoder block. We conduct extensive evaluations on long- and short-term forecasting datasets comparing MultiResFormer with state-of-the-art baselines. MultiResFormer outperforms patch-based Transformer baselines on long-term forecasting tasks and also consistently outperforms CNN baselines by a large margin, while using much fewer parameters than these baselines.