Using dynamic loss weighting to boost improvements in forecast stability

TL;DR

This paper explores dynamic loss weighting techniques to enhance forecast stability in time series models, demonstrating that adaptive weighting can improve stability without sacrificing accuracy.

Contribution

It introduces and empirically evaluates dynamic loss weighting algorithms, including a novel Task-Aware Random Weighting method, to improve forecast stability.

Findings

Dynamic loss weighting improves forecast stability.

Existing methods can enhance stability without accuracy loss.

Task-Aware Random Weighting is a new effective approach.

Abstract

Rolling origin forecast instability refers to variability in forecasts for a specific period induced by updating the forecast when new data points become available. Recently, an extension to the N-BEATS model for univariate time series point forecasting was proposed to include forecast stability as an additional optimization objective, next to accuracy. It was shown that more stable forecasts can be obtained without harming accuracy by minimizing a composite loss function that contains both a forecast error and a forecast instability component, with a static hyperparameter to control the impact of stability. In this paper, we empirically investigate whether further improvements in stability can be obtained without compromising accuracy by applying dynamic loss weighting algorithms, which change the loss weights during training. We show that existing dynamic loss weighting methods can achieve this objective and provide insights into why this might be the case. Additionally, we propose an extension to the Random Weighting approach -- Task-Aware Random Weighting -- which also achieves this objective.