Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting

TL;DR

Pathformer is a novel multi-scale Transformer model for time series forecasting that adaptively captures diverse temporal characteristics, leading to superior accuracy and generalization across multiple real-world datasets.

Contribution

It introduces adaptive pathways in multi-scale Transformers, enabling dynamic adjustment to temporal variations, which improves forecasting performance and robustness.

Findings

Achieves state-of-the-art results on eleven datasets.

Demonstrates strong generalization under transfer scenarios.

Outperforms existing models in accuracy and adaptability.

Abstract

Transformers for time series forecasting mainly model time series from limited or fixed scales, making it challenging to capture different characteristics spanning various scales. We propose Pathformer, a multi-scale Transformer with adaptive pathways. It integrates both temporal resolution and temporal distance for multi-scale modeling. Multi-scale division divides the time series into different temporal resolutions using patches of various sizes. Based on the division of each scale, dual attention is performed over these patches to capture global correlations and local details as temporal dependencies. We further enrich the multi-scale Transformer with adaptive pathways, which adaptively adjust the multi-scale modeling process based on the varying temporal dynamics of the input, improving the accuracy and generalization of Pathformer. Extensive experiments on eleven real-world datasets demonstrate that Pathformer not only achieves state-of-the-art performance by surpassing all current models but also exhibits stronger generalization abilities under various transfer scenarios. The code is made available at https://github.com/decisionintelligence/pathformer.