Ensemble size effects on conditional reliability estimates: slope attenuation bias and correction methods

TL;DR

This paper investigates how finite ensemble sizes bias conditional reliability diagnostics in ensemble forecasting, deriving correction methods and illustrating their importance with temperature forecast data.

Contribution

It introduces a unified framework for understanding and correcting slope attenuation bias caused by finite ensemble sampling noise in reliability assessments.

Findings

Finite ensemble sizes systematically bias conditional reliability diagnostics.

Analytical expressions for attenuation are derived and practical estimators are proposed.

Finite-ensemble effects significantly impact spread-error relationships and reliability diagrams.

Abstract

The goal of ensemble forecasting is to maximise sharpness subject to reliability. Marginal reliability means that, over all cases, the ensemble is statistically consistent with reality: the ensemble mean is unbiased, the expected ensemble variance equals the expected mean-squared error of the ensemble mean, and the variance of the ensemble members matches the variance of the truth. Equivalently, forecasts that assign probability $p$ to an event verify with relative frequency $p$. However, climatological consistency is not sufficient for users acting on individual forecasts. A natural extension is to assess reliability conditional on the forecast itself, by examining whether, on average, larger ensemble means imply larger observed values, larger spreads imply larger forecast errors, or higher probabilities imply higher event frequencies. This motivates conditional reliability diagnostics such as reliability diagrams and spread-error relationships. Here we show that conditional reliability diagnostics are systematically biased for finite ensemble sizes. We present a unified framework for slope attenuation caused by finite-ensemble sampling noise, which affects conditional diagnostics for ensemble means, spreads, and probabilities. Using synthetic forecasts that are perfectly reliable by construction, we isolate finite-ensemble effects. We derive analytical expressions for the expected attenuation and propose practical estimators computable directly from ensemble data. The framework is illustrated using 2-metre temperature sub-seasonal ensemble forecasts from ECMWF, where finite-ensemble slope attenuation substantially affects the spread-error relationship and tercile-based reliability diagrams. These results demonstrate that attenuated conditional slopes should not be interpreted as evidence of forecast deficiencies unless finite-ensemble effects are explicitly taken into account.