FFORMPP: Feature-based forecast model performance prediction

TL;DR

This paper presents FFORMPP, a meta-learning framework that predicts time series forecast performance using features and Bayesian regression, augmented with simulation data, achieving comparable results with lower computational cost and better interpretability.

Contribution

The paper introduces a novel meta-learning algorithm for forecast performance prediction that combines feature-based modeling, Bayesian regression, and simulation data augmentation.

Findings

Comparable performance to existing methods

Lower computational cost

Enhanced interpretability

Abstract

This paper introduces a novel meta-learning algorithm for time series forecast model performance prediction. We model the forecast error as a function of time series features calculated from the historical time series with an efficient Bayesian multivariate surface regression approach. The minimum predicted forecast error is then used to identify an individual model or a combination of models to produce the final forecasts. It is well-known that the performance of most meta-learning models depends on the representativeness of the reference dataset used for training. In such circumstances, we augment the reference dataset with a feature-based time series simulation approach, namely GRATIS, in generating a rich and representative time series collection. The proposed framework is tested using the M4 competition data and is compared against commonly used forecasting approaches. Our approach provides comparable performances to other model selection/combination approaches but at a lower computational cost and a higher degree of interpretability, which is important for supporting decisions. We also provide useful insights regarding which forecasting models are expected to work better for particular types of time series, the intrinsic mechanisms of the meta-learners and how the forecasting performances are affected by various factors.