A Tolerance-Based Framework for Spatio-Temporal Forecast Validation Using the gamma-Index

TL;DR

This paper introduces a gamma-index based framework for spatio-temporal forecast validation that accounts for physical tolerances, addressing limitations of traditional point-wise metrics like RMSE.

Contribution

It adapts the gamma index from medical physics to forecast verification, enabling tolerance-based assessment of gridded predictions in space, time, and intensity.

Findings

Gamma criterion preserves structural consistency with minor positional noise.

Application to satellite data demonstrates operational effectiveness.

Framework is generic and adaptable to various gridded variables.

Abstract

Classical field forecast evaluation relies mainly on local scores such as RMSE or MAE. These metrics severely over-penalize small spatial or temporal displacements of coherent structures, a limitation known as the double-penalty issue and common to many forecasting domains. The present paper introduces a tolerance-based framework built on the three-dimensional gamma index, initially designed for medical dose verification, as a unified acceptance criterion for gridded forecasts. The method embeds explicit margins in space (DTA), time (TTA), and intensity (IDT), and evaluates whether predictions agree with observations within predefined physical bounds rather than through pixel-wise differences only. A synthetic illustration is first used to show why conventional metrics can misrepresent usable forecasts. The approach is then applied to satellite-derived SSI fields to demonstrate operational behaviour on a real dataset. Results confirm that the gamma criterion preserves structural consistency under minor positional noise while isolating physically significant discrepancies. The formulation is generic and can be implemented for any gridded variable provided meaningful tolerances are defined, offering a pragmatic complement to existing spatial verification tools in general forecasting workflows.