More on verification of probability forecasts for football outcomes: score decompositions, reliability, and discrimination analyses

TL;DR

This paper introduces advanced verification tools for football outcome probability forecasts, including score decompositions and reliability analyses, demonstrated on UEFA Champions League data, enhancing forecast assessment beyond mean scores.

Contribution

It proposes new graphical and numerical verification methods, such as score decompositions and reliability diagnostics, for evaluating football forecast quality.

Findings

Forecasts showed good reliability compared to bookmaker odds.

Score decompositions provided insights into forecast components.

Methods are applicable to other fields like meteorology and medicine.

Abstract

Forecast of football outcomes in terms of Home Win, Draw and Away Win relies largely on ex ante probability elicitation of these events and ex post verification of them via computation of probability scoring rules (Brier, Ranked Probability, Logarithmic, Zero-One scores). Usually, appraisal of the quality of forecasting procedures is restricted to reporting mean score values. The purpose of this article is to propose additional tools of verification, such as score decompositions into several components of special interest. Graphical and numerical diagnoses of reliability and discrimination and kindred statistical methods are presented using different techniques of binning (fixed thresholds, quantiles, logistic and iso regression). These procedures are illustrated on probability forecasts for the outcomes of the UEFA Champions League (C1) at the end of the group stage based on typical Poisson regression models with reasonably good results in terms of reliability as compared to those obtained from bookmaker odds and whatever the technique used. Links with research in machine learning and different areas of application (meteorology, medicine) are discussed.