Proper Scoring Rules for Multivariate Probabilistic Forecasts based on Aggregation and Transformation

TL;DR

This paper introduces a flexible framework for creating interpretable multivariate proper scoring rules through aggregation and transformation, enhancing the assessment of probabilistic forecasts by targeting specific features.

Contribution

It formalizes a novel framework that combines aggregation and transformation to develop scoring rules that better characterize multivariate forecast performance.

Findings

Framework improves interpretability of multivariate scoring rules

Numerical experiments demonstrate benefits over traditional methods

Bridges gap between proper scoring rules and spatial verification tools

Abstract

Proper scoring rules are an essential tool to assess the predictive performance of probabilistic forecasts. However, propriety alone does not ensure an informative characterization of predictive performance and it is recommended to compare forecasts using multiple scoring rules. With that in mind, interpretable scoring rules providing complementary information are necessary. We formalize a framework based on aggregation and transformation to build interpretable multivariate proper scoring rules. Aggregation-and-transformation-based scoring rules are able to target specific features of the probabilistic forecasts; which improves the characterization of the predictive performance. This framework is illustrated through examples taken from the literature and studied using numerical experiments showcasing its benefits. In particular, it is shown that it can help bridge the gap between proper scoring rules and spatial verification tools.