Kelly Betting as Bayesian Model Evaluation: A Framework for Time-Updating Probabilistic Forecasts

TL;DR

This paper introduces a novel Bayesian-inspired framework for evaluating time-updating probabilistic forecasts by treating models as Kelly bettors, using bankroll growth as a real-time accuracy metric, which outperforms traditional methods.

Contribution

It presents a new evaluation approach that treats probabilistic models as Kelly bettors, enabling real-time credibility updates and improved model discrimination.

Findings

The Kelly-based method outperforms traditional log-loss and Brier score in simulations.

Bankroll growth effectively measures model accuracy over time.

The approach provides a Bayesian analogue for model credibility assessment.

Abstract

This paper proposes a new way of evaluating the accuracy and validity of probabilistic forecasts that change over time (such as an in-game win probability model, or an election forecast). Under this approach, each model to be evaluated is treated as a canonical Kelly bettor, and the models are pitted against each other in an iterative betting contest. The growth or decline of each model's bankroll serves as the evaluation metric. Under this approach, market consensus probabilities and implied model credibilities can be updated real time as each model updates, and do not require one to wait for the final outcome. Using a simulation model, it will be shown that this method is in general more accurate than traditional average log-loss and Brier score methods at distinguishing a correct model from an incorrect model. This Kelly approach is shown to have a direct mathematical and conceptual analogue to Bayesian inference, with bankroll serving as a proxy for Bayesian credibility.