Does "model-free" forecasting really outperform the "true" model? A reply to Perretti et al

TL;DR

This paper challenges the claim that model-free forecasting always outperforms true models by showing that bias in Bayesian state-space methods can be corrected, restoring the advantage of mechanistic models.

Contribution

It demonstrates that bias in Bayesian state-space estimates can be mitigated, reversing previous conclusions about the superiority of model-free forecasting in ecological data.

Findings

Bias in Bayesian estimates affects forecast accuracy.

Correcting bias restores the advantage of true mechanistic models.

Model-free methods are not inherently superior when bias is addressed.

Abstract

Estimating population models from uncertain observations is an important problem in ecology. Perretti et al. observed that standard Bayesian state-space solutions to this problem may provide biased parameter estimates when the underlying dynamics are chaotic. Consequently, forecasts based on these estimates showed poor predictive accuracy compared to simple "model-free" methods, which lead Perretti et al. to conclude that "Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data". However, a simple modification of the statistical methods also suffices to remove the bias and reverse their results.