Stratify: Unifying Multi-Step Forecasting Strategies

TL;DR

Stratify introduces a unified, parameterized framework for multi-step forecasting, enabling the comparison and improvement of strategies across diverse datasets and horizons, emphasizing the importance of strategy selection tailored to specific tasks.

Contribution

We propose Stratify, a comprehensive framework that unifies and enhances multi-step forecasting strategies, providing a systematic approach for strategy selection and benchmarking.

Findings

Novel strategies improved performance in over 84% of experiments.

No single strategy outperforms others across all tasks.

Benchmarking shows the importance of task-specific strategy selection.

Abstract

A key aspect of temporal domains is the ability to make predictions multiple time steps into the future, a process known as multi-step forecasting (MSF). At the core of this process is selecting a forecasting strategy, however, with no existing frameworks to map out the space of strategies, practitioners are left with ad-hoc methods for strategy selection. In this work, we propose Stratify, a parameterised framework that addresses multi-step forecasting, unifying existing strategies and introducing novel, improved strategies. We evaluate Stratify on 18 benchmark datasets, five function classes, and short to long forecast horizons (10, 20, 40, 80). In over 84% of 1080 experiments, novel strategies in Stratify improved performance compared to all existing ones. Importantly, we find that no single strategy consistently outperforms others in all task settings, highlighting the need for practitioners explore the Stratify space to carefully search and select forecasting strategies based on task-specific requirements. Our results are the most comprehensive benchmarking of known and novel forecasting strategies. We make code available to reproduce our results.