Multi-layer Stack Ensembles for Time Series Forecasting

TL;DR

This paper explores ensemble methods for time series forecasting, demonstrating that multi-layer stacking consistently enhances accuracy across diverse datasets and proposing a novel framework to combine different stacker models.

Contribution

It introduces a multi-layer stacking framework for time series forecasting, improving accuracy and addressing the limitations of single-layer ensembles.

Findings

Multi-layer stacking outperforms single-layer methods across datasets

Stacking improves forecasting accuracy consistently

Proposed framework combines strengths of different models

Abstract

Ensembling is a powerful technique for improving the accuracy of machine learning models, with methods like stacking achieving strong results in tabular tasks. In time series forecasting, however, ensemble methods remain underutilized, with simple linear combinations still considered state-of-the-art. In this paper, we systematically explore ensembling strategies for time series forecasting. We evaluate 33 ensemble models -- both existing and novel -- across 50 real-world datasets. Our results show that stacking consistently improves accuracy, though no single stacker performs best across all tasks. To address this, we propose a multi-layer stacking framework for time series forecasting, an approach that combines the strengths of different stacker models. We demonstrate that this method consistently provides superior accuracy across diverse forecasting scenarios. Our findings highlight the potential of stacking-based methods to improve AutoML systems for time series forecasting.