Testing long-term earthquake forecasts: likelihood methods and error diagrams

TL;DR

This paper introduces a new likelihood-based method for testing long-term earthquake forecasts, providing a more accurate and less simulation-dependent evaluation tool that includes error analysis and visualization techniques.

Contribution

It presents a novel likelihood scoring approach and error diagram visualization for assessing earthquake forecast effectiveness, applicable to spatial point process predictions.

Findings

The method effectively evaluates seismic forecasts in Pacific regions.

Forecast score distribution approaches Gaussian with more events.

Error diagrams can be biased with small event samples.

Abstract

We propose a new method to test the effectiveness of a spatial point process forecast based on a log-likelihood score for predicted point density and the information gain for events that actually occurred in the test period. The method largely avoids simulation use and allows us to calculate the information score for each event or set of events as well as the standard error of each forecast. As the number of predicted events increases, the score distribution approaches the Gaussian law. The degree of its similarity to the Gaussian distribution can be measured by the computed coefficients of skewness and kurtosis. To display the forecasted point density and the point events, we use an event concentration diagram or a variant of the Error Diagram (ED). We demonstrate the application of the method by using our long-term forecast of seismicity in two western Pacific regions. We compare the ED for these regions with simplified diagrams based on two-segment approximations. Since the earthquakes in these regions are concentrated in narrow subduction belts, using the forecast density as a template or baseline for the ED is a more convenient display technique. We also show, using simulated event occurrence, that some proposed criteria for measuring forecast effectiveness at EDs would be strongly biased for a small event number.