Locally tail-scale invariant scoring rules for evaluation of extreme value forecasts

TL;DR

This paper introduces local tail-scale invariant scoring rules for evaluating extreme value forecasts, including a new scaled version of the CRPS, to better assess models in regions with varying extreme event scales.

Contribution

It proposes the concept of local weight-scale invariance and develops the scaled wCRPS score, providing a tailored evaluation method for extreme value models in diverse regions.

Findings

The scaled wCRPS score is effective for extreme event evaluation.

Explicit formulas for the score are derived for GEV distributions.

Simulation and real-world applications demonstrate the score's usefulness.

Abstract

Statistical analysis of extremes can be used to predict the probability of future extreme events, such as large rainfalls or devastating windstorms. The quality of these forecasts can be measured through scoring rules. Locally scale invariant scoring rules give equal importance to the forecasts at different locations regardless of differences in the prediction uncertainty. This is a useful feature when computing average scores but can be an unnecessarily strict requirement when mostly concerned with extremes. We propose the concept of local weight-scale invariance, describing scoring rules fulfilling local scale invariance in a certain region of interest, and as a special case local tail-scale invariance, for large events. Moreover, a new version of the weighted Continuous Ranked Probability score (wCRPS) called the scaled wCRPS (swCRPS) that possesses this property is developed and studied. The score is a suitable alternative for scoring extreme value models over areas with varying scale of extreme events, and we derive explicit formulas of the score for the Generalised Extreme Value distribution. The scoring rules are compared through simulation, and their usage is illustrated in modelling of extreme water levels, annual maximum rainfalls, and in an application to non-extreme forecast for the prediction of air pollution.