Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution

TL;DR

This paper introduces Conformer, an efficient Transformer-based model for long-term time-series forecasting that enhances information utilization and modeling of complex dependencies, outperforming existing methods on multiple real-world datasets.

Contribution

The paper proposes a novel Conformer model with an encoder-decoder architecture, flow-based latent inference, and explicit modeling of inter-series correlation and dynamics for improved long-term forecasting.

Findings

Conformer achieves superior accuracy on seven real-world datasets.

It provides reliable predictions with uncertainty quantification.

The model effectively captures complex dependencies in time-series data.

Abstract

Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. Though one could lower the complexity of Transformers by inducing the sparsity in point-wise self-attentions for LTTF, the limited information utilization prohibits the model from exploring the complex dependencies comprehensively. To this end, we propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects: (i) an encoder-decoder architecture incorporating a linear complexity without sacrificing information utilization is proposed on top of sliding-window attention and Stationary and Instant Recurrent Network (SIRN); (ii) a module derived from the normalizing flow is devised to further improve the information utilization by inferring the outputs with the latent variables in SIRN directly; (iii) the inter-series correlation and temporal dynamics in time-series data are modeled explicitly to fuel the downstream self-attention mechanism. Extensive experiments on seven real-world datasets demonstrate that Conformer outperforms the state-of-the-art methods on LTTF and generates reliable prediction results with uncertainty quantification.