The ecological forecast limit revisited: Potential, actual and relative system predictability

TL;DR

This paper reviews and unifies methods for determining ecological forecast limits, enabling better assessment of how far into the future ecological predictions can be trusted across various models and systems.

Contribution

It introduces a unified framework for empirical ecological forecast limits, distinguishing potential, absolute, and relative limits, with practical computation recipes and case studies.

Findings

Forecast limits require a verification reference, scoring function, and error tolerance.

The framework applies across population, ecosystem, and Earth system models.

Benchmark models help determine the relative forecast limit.

Abstract

Ecological forecasts are model-based statements about currently unknown ecosystem states in time or space. For a model forecast to be useful to inform decision makers, model validation and verification determine adequateness. The measure of forecast goodness that can be translated into a limit up to which a forecast is acceptable is known as the 'forecast limit'. While verification in weather forecasting follows strict criteria with established metrics and forecast limits, assessments of ecological forecasting models still remain experiment-specific, and forecast limits are rarely reported. As such, users of ecological forecasts remain uninformed of how far into the future statements can be trusted. In this work, we synthesise existing approaches to define empirical forecast limits in a unified framework for assessing ecological predictability and offer recipes for their computation. We distinguish the model's potential and absolute forecast limit, and show how a benchmark model can help determine its relative forecast limit. The approaches are demonstrated with three case studies from population, ecosystem, and Earth system research. We found that forecast limits can be computed with three requirements: A verification reference, a scoring function, and a predictive error tolerance. Within our framework, forecast limits are defined for practically any ecological forecast and support research on ecological predictability analysis.