Predicting Failures of Point Forecasts

TL;DR

This paper investigates the predictability of errors in temperature forecasts by analyzing ensemble outputs to generate warnings for significant deviations, evaluating prediction quality with various scores, and examining the predictability of rare events.

Contribution

It introduces a decision-making approach to forecast errors using ensemble data and evaluates its effectiveness with multiple statistical metrics, focusing on rare event predictability.

Findings

Forecast errors can be predicted with measurable skill.

Ensemble analysis improves warning accuracy.

Rare events are more challenging but potentially more predictable.

Abstract

The predictability of errors in deterministic temperature forecasts is investigated. More precisely, the aim is to issue warnings whenever the differences between forecast and verification exceed a given threshold. The warnings are generated by analyzing the output of an ensemble forecast system in terms of a decision making approach. The quality of the resulting predictions is evaluated by computing receiver operating characteristics, the Brier score, and the Ignorance score. Special emphasis is also given to the question whether rare events are better predictable.