A Review of the Long Horizon Forecasting Problem in Time Series Analysis

TL;DR

This review comprehensively discusses the evolution and recent advances in long horizon forecasting in time series analysis, emphasizing deep learning techniques, model architectures, and data preprocessing methods that enhance forecasting accuracy.

Contribution

It provides a detailed overview of deep learning innovations and techniques specifically tailored for long horizon forecasting, highlighting recent model architectures and preprocessing strategies.

Findings

Error increases with forecast horizon, except for xLSTM and Triformer models.

Deep learning models improve long horizon forecasts through advanced architectures.

Ablation studies demonstrate the effectiveness of various techniques on the ETTm2 dataset.

Abstract

The long horizon forecasting (LHF) problem has come up in the time series literature for over the last 35 years or so. This review covers aspects of LHF in this period and how deep learning has incorporated variants of trend, seasonality, fourier and wavelet transforms, misspecification bias reduction and bandpass filters while contributing using convolutions, residual connections, sparsity reduction, strided convolutions, attention masks, SSMs, normalization methods, low-rank approximations and gating mechanisms. We highlight time series decomposition techniques, input data preprocessing and dataset windowing schemes that improve performance. Multi-layer perceptron models, recurrent neural network hybrids, self-attention models that improve and/or address the performances of the LHF problem are described, with an emphasis on the feature space construction. Ablation studies are conducted over the ETTm2 dataset in the multivariate and univariate high useful load (HUFL) forecasting contexts, evaluated over the last 4 months of the dataset. The heatmaps of MSE averages per time step over test set series in the horizon show that there is a steady increase in the error proportionate to its length except with xLSTM and Triformer models and motivate LHF as an error propagation problem. The trained models are available here: https://bit.ly/LHFModelZoo