Non-asymptotic calibration and resolution

TL;DR

This paper introduces a new data-driven probability forecasting algorithm for binary data that guarantees calibration and resolution with explicit performance bounds, applicable to long sequences without assumptions on data generation.

Contribution

It presents a non-asymptotic analysis of a novel calibration algorithm with explicit performance bounds, improving understanding of probabilistic forecasting methods.

Findings

Algorithm is well calibrated and has good resolution for long sequences.

Explicit, tight inequalities for the algorithm's performance are established.

Performance depends on the choice of a kernel on forecast and data spaces.

Abstract

We analyze a new algorithm for probability forecasting of binary observations on the basis of the available data, without making any assumptions about the way the observations are generated. The algorithm is shown to be well calibrated and to have good resolution for long enough sequences of observations and for a suitable choice of its parameter, a kernel on the Cartesian product of the forecast space $[0,1]$ and the data space. Our main results are non-asymptotic: we establish explicit inequalities, shown to be tight, for the performance of the algorithm.