An Adaptive Horizon-Aware Model Selection Framework for Demand Forecasting under Horizon-Induced Degradation

TL;DR

This paper introduces a horizon-aware model selection framework for demand forecasting, addressing challenges of demand variability and multi-step planning by projecting error metrics to future horizons.

Contribution

It proposes the MDFH procedure and the RMSSEh metric, along with the AHSIV selector, to improve model selection for complex, intermittent demand environments.

Findings

MDFH provides a coherent horizon-aware evaluation basis.

RMSSEh and AHSIV are competitive across diverse demand scenarios.

AHSIV enhances robustness in complex, variable demand settings.

Abstract

Business environments characterized by intermittent demand, high variability, and multi-step planning require model selection procedures aligned with future operational horizons rather than static test-horizon evaluation. Because no forecasting model is universally dominant, and rankings vary across metrics, demand structures, and forecast horizons, assigning an appropriate model to each series remains a difficult problem in inventory planning, procurement, and supply management. This study addresses that problem by introducing the Metric Degradation by Forecast Horizon (MDFH) procedure as its main methodological contribution. MDFH projects out-of-sample error metrics from the test horizon to a future operational horizon under structural stability conditions, converting conventional static evaluation into a horizon-aware scheme for multi-step decision contexts. From this basis, the study derives RMSSEh as the most parsimonious operational realization of MDFH and proposes the Adaptive Hybrid Selector for Intermittency and Variability (AHSIV) as an adaptive extension for cases where monometric horizon-aware selection is insufficient due to intermittency, variability, metric conflict, and forecast bias. Empirical evaluation on the Walmart, M3, M4, and M5 datasets, using multiple train-test partitions and 12-step forecasting horizons, compares RMSSEh, AHSIV, and ERA as selector mechanisms. Results show that MDFH provides a coherent basis for horizon-aware selector design, that RMSSEh and AHSIV remain competitive across heterogeneous demand environments, and that AHSIV adds robustness in structurally complex settings. Overall, forecasting model selection in multi-SKU environments should be treated as a horizon-aware, structure-sensitive assignment problem aligned with operational planning requirements.