Probabilistic Decomposition Transformer for Time Series Forecasting

TL;DR

This paper introduces a probabilistic decomposition Transformer that combines sequence modeling with generative techniques to produce hierarchical, interpretable, and non-autoregressive forecasts for complex time series, improving accuracy and robustness.

Contribution

It presents a novel model integrating Transformers with a conditional generative approach for hierarchical probabilistic time series forecasting, addressing limitations of autoregressive models.

Findings

Outperforms state-of-the-art methods on multiple datasets

Provides interpretable forecasts by reconstructing seasonality and trend

Demonstrates robustness and effectiveness in complex scenarios

Abstract

Time series forecasting is crucial for many fields, such as disaster warning, weather prediction, and energy consumption. The Transformer-based models are considered to have revolutionized the field of sequence modeling. However, the complex temporal patterns of the time series hinder the model from mining reliable temporal dependencies. Furthermore, the autoregressive form of the Transformer introduces cumulative errors in the inference step. In this paper, we propose the probabilistic decomposition Transformer model that combines the Transformer with a conditional generative model, which provides hierarchical and interpretable probabilistic forecasts for intricate time series. The Transformer is employed to learn temporal patterns and implement primary probabilistic forecasts, while the conditional generative model is used to achieve non-autoregressive hierarchical probabilistic forecasts by introducing latent space feature representations. In addition, the conditional generative model reconstructs typical features of the series, such as seasonality and trend terms, from probability distributions in the latent space to enable complex pattern separation and provide interpretable forecasts. Extensive experiments on several datasets demonstrate the effectiveness and robustness of the proposed model, indicating that it compares favorably with the state of the art.