Epistemic Error Decomposition for Multi-step Time Series Forecasting: Rethinking Bias-Variance in Recursive and Direct Strategies

TL;DR

This paper revisits the bias-variance trade-off in multi-step time series forecasting, decomposing forecast errors to provide nuanced insights into recursive and direct strategies, especially for nonlinear models.

Contribution

It introduces a new error decomposition framework that clarifies the bias-variance dynamics in recursive versus direct forecasting, challenging traditional beliefs.

Findings

Recursive strategies can have lower bias and higher variance depending on model nonlinearity.

Structural approximation gap is zero for linear predictors but significant for nonlinear ones.

Guidance for choosing forecasting strategies based on model and data characteristics.

Abstract

Multi-step forecasting is often described through a simple rule of thumb: recursive strategies are said to have high bias and low variance, while direct strategies are said to have low bias and high variance. We revisit this belief by decomposing the expected multi-step forecast error into three parts: irreducible noise, a structural approximation gap, and an estimation-variance term. For linear predictors we show that the structural gap is identically zero for any dataset. For nonlinear predictors, however, the repeated composition used in recursion can increase model expressivity, making the structural gap depend on both the model and the data. We further show that the estimation variance of the recursive strategy at any horizon can be written as the one-step variance multiplied by a Jacobian-based amplification factor that measures how sensitive the composed predictor is to parameter error. This perspective explains when recursive forecasting may simultaneously have lower bias and higher variance than direct forecasting. Experiments with multilayer perceptrons on the ETTm1 dataset confirm these findings. The results offer practical guidance for choosing between recursive and direct strategies based on model nonlinearity and noise characteristics, rather than relying on traditional bias-variance intuition.