Ecosystem models cannot predict the consequences of conservation decisions

TL;DR

Ecosystem models, despite calibration to experimental data, often fail to accurately predict future ecosystem responses to management interventions, raising doubts about their utility in applied ecology decisions.

Contribution

This study empirically evaluates the predictive accuracy of ecosystem models calibrated with microcosm data, highlighting their limitations in forecasting management outcomes.

Findings

Many parameter sets fit the data equally well.

Models have poor predictive accuracy for future dynamics.

Calibration cannot reliably identify ecosystem interactions.

Abstract

Ecosystem models are often used to predict the consequences of management decisions in applied ecology, including fisheries management and threatened species conservation. These models are high-dimensional, parameter-rich, and nonlinear, yet limited data is available to calibrate them, and they are rarely tested or validated. Consequently, the accuracy of their forecasts, and their utility as decision-support tools is a matter of debate. In this paper, we calibrate ecosystem models to time-series data from 110 different experimental microcosm ecosystems, each containing between three and five interacting species. We then assess how often these calibrated models offer accurate and useful predictions about how the ecosystem will respond to a set of standard management interventions. Our results show that for each timeseries dataset, a large number of very different parameter sets offer equivalent, good fits. However, these calibrated ecosystem models have poor predictive accuracy when forecasting future dynamics and offer ambiguous predictions about how species in the ecosystem will respond to management interventions. Closer inspection reveals that the ecosystem models fail because calibration cannot determine the types of interactions that occur within the ecosystem. Our findings call into question claims that ecosystem modelling can support applied ecological decision-making when they are calibrated against real-world datasets.