A Bayesian framework for verification and recalibration of ensemble forecasts: How uncertain is NAO predictability?

TL;DR

This paper introduces a Bayesian framework to quantify uncertainty and improve calibration in ensemble forecasts, specifically applied to North Atlantic Oscillation predictions, enhancing understanding of forecast skill and reliability.

Contribution

It presents a novel Bayesian inferential model that quantifies uncertainty and enables recalibration of ensemble forecasts, addressing limitations of traditional predictability estimates.

Findings

Strong evidence of forecast skill with correlation credible interval [0.19,0.68]

Forecast signal-to-noise ratio is significantly lower than that of observations

Recalibration improves forecast reliability and interpretability

Abstract

Predictability estimates of ensemble prediction systems are uncertain due to limited numbers of past forecasts and observations. To account for such uncertainty, this paper proposes a Bayesian inferential framework that provides a simple 6-parameter representation of ensemble forecasting systems and the corresponding observations. The framework is probabilistic, and thus allows for quantifying uncertainty in predictability measures such as correlation skill and signal-to-noise ratios. It also provides a natural way to produce recalibrated probabilistic predictions from uncalibrated ensembles forecasts. The framework is used to address important questions concerning the skill of winter hindcasts of the North Atlantic Oscillation for 1992-2011 issued by the Met Office GloSea5 climate prediction system. Although there is much uncertainty in the correlation between ensemble mean and observations, there is strong evidence of skill: the 95% credible interval of the correlation coefficient of [0.19,0.68] does not overlap zero. There is also strong evidence that the forecasts are not exchangeable with the observations: With over 99% certainty, the signal-to-noise ratio of the forecasts is smaller than the signal-to-noise ratio of the observations, which suggests that raw forecasts should not be taken as representative scenarios of the observations. Forecast recalibration is thus required, which can be coherently addressed within the proposed framework.