Comparative evaluation of point process forecasts

TL;DR

This paper adapts proper scoring rules for evaluating point process forecasts, demonstrating their broad applicability and providing a principled framework for comparing models, exemplified through earthquake forecast evaluation.

Contribution

It introduces a unified framework for the evaluation of point process forecasts using proper scoring rules, extending existing methods to broader contexts.

Findings

Poisson log-likelihood is effective for forecast comparison.

Proper scoring rules can be applied to diverse point process models.

The approach is validated through simulations and earthquake forecast data.

Abstract

Stochastic models of point patterns in space and time are widely used to issue forecasts or assess risk, and often they affect societally relevant decisions. We adapt the concept of consistent scoring functions and proper scoring rules, which are statistically principled tools for the comparative evaluation of predictive performance, to the point process setting, and place both new and existing methodology in this framework. With reference to earthquake likelihood model testing, we demonstrate that extant techniques apply in much broader contexts than previously thought. In particular, the Poisson log-likelihood can be used for theoretically principled comparative forecast evaluation in terms of cell expectations. We illustrate the approach in a simulation study and in a comparative evaluation of operational earthquake forecasts for Italy.