Late Meta-learning Fusion Using Representation Learning for Time Series Forecasting

TL;DR

This paper introduces a unified taxonomy for model fusion in time series forecasting and proposes DeFORMA, a novel hybrid approach that achieves state-of-the-art results on the M4 dataset by combining meta-learning and representation learning techniques.

Contribution

It provides the first comprehensive taxonomy for model fusion methods in time series forecasting and empirically evaluates a new hybrid approach, DeFORMA, demonstrating its superior performance.

Findings

DeFORMA outperforms existing methods on the M4 dataset.

The taxonomy clarifies the relationships among hybrid and feature-based ensemble methods.

DeFORMA achieves significant improvements in daily, weekly, and yearly forecast subsets.

Abstract

Meta-learning, decision fusion, hybrid models, and representation learning are topics of investigation with significant traction in time-series forecasting research. Of these two specific areas have shown state-of-the-art results in forecasting: hybrid meta-learning models such as Exponential Smoothing - Recurrent Neural Network (ES-RNN) and Neural Basis Expansion Analysis (N-BEATS) and feature-based stacking ensembles such as Feature-based FORecast Model Averaging (FFORMA). However, a unified taxonomy for model fusion and an empirical comparison of these hybrid and feature-based stacking ensemble approaches is still missing. This study presents a unified taxonomy encompassing these topic areas. Furthermore, the study empirically evaluates several model fusion approaches and a novel combination of hybrid and feature stacking algorithms called Deep-learning FORecast Model Averaging (DeFORMA). The taxonomy contextualises the considered methods. Furthermore, the empirical analysis of the results shows that the proposed model, DeFORMA, can achieve state-of-the-art results in the M4 data set. DeFORMA, increases the mean Overall Weighted Average (OWA) in the daily, weekly and yearly subsets with competitive results in the hourly, monthly and quarterly subsets. The taxonomy and empirical results lead us to argue that significant progress is still to be made by continuing to explore the intersection of these research areas.