Reply to Hartig et al. [arXiv:1305.3544]: The "true model" myth

TL;DR

This paper critiques a model-fitting method for ecological systems, demonstrating its limitations in real-world scenarios where the true model is unknown, and advocates for a model-free approach for better accuracy and understanding.

Contribution

The paper challenges the effectiveness of a proposed model-fitting method in unknown true model conditions and promotes a model-free approach for ecological forecasting.

Findings

Model-fitting methods fail with chaotic systems when the true model is unknown.

The suggested method becomes inaccurate with small modifications to the true model.

A model-free approach provides more accurate forecasts and better ecological insights.

Abstract

We have shown that commonly used model-fitting methods fail when the underlying ecological system is chaotic. Importantly, Hartig & Dormann [arXiv:1305.3544] suggest an alternative method that is able to overcome this problem for the logistic model when the "true model" is known. This is encouraging, however the more relevant test is of the method when the "true model" is unknown, as it always is for real ecosystems. By making a small modification to the "true model," we find that the suggested method provides inaccurate forecasts and severely biased estimates of the ecological dynamics. In contrast, a model-free approach is able to provide accurate forecasts and a useful foundation for ecological understanding without the burdensome assumption of a "true model."