Skill of data based predictions versus dynamical models -- case study on extreme temperature anomalies

TL;DR

This study compares the predictive skill of a complex dynamical weather model and a simple data model for extreme temperature anomalies, revealing that the simpler model can outperform the complex one under certain conditions, depending on evaluation methods.

Contribution

It demonstrates that simple data models can achieve competitive predictive skill for extreme temperature anomalies and highlights the importance of evaluation metrics.

Findings

Simple data model performs well in some parameter ranges.

Performance varies significantly with evaluation method.

Complex weather model generally outperforms the simple model.

Abstract

We compare probabilistic predictions of extreme temperature anomalies issued by two different forecast schemes. One is a dynamical physical weather model, the other a simple data model. We recall the concept of skill scores in order to assess the performance of these two different predictors. Although the result confirms the expectation that the (computationally expensive) weather model outperforms the simple data model, the performance of the latter is surprisingly good. More specifically, for some parameter range, it is even better than the uncalibrated weather model. Since probabilistic predictions are not easily interpreted by the end user, we convert them into deterministic yes/no statements and measure the performance of these by ROC statistics. Scored in this way, conclusions about model performance partly change, which illustrates that predictive power depends on how it is quantified.