Kairosis: A method for dynamical probability forecast aggregation informed by Bayesian change point detection

TL;DR

Kairosis is a novel Bayesian change-point informed method for aggregating time-series probability forecasts, emphasizing recent forecast changes to improve accuracy in dynamic environments.

Contribution

The paper introduces kairosis, a new approach that dynamically weights forecasts based on detected change points, enhancing aggregation accuracy over traditional methods.

Findings

Kairosis outperforms standard aggregation methods.

It is robust across diverse forecasting scenarios.

Effective in geopolitical forecasting contexts.

Abstract

We present a new method, "kairosis", for aggregating probability forecasts made over a time period of a single outcome determined at the end of that period. Informed by work on Bayesian change-point detection, we begin by constructing for each time during the period a posterior probability that the forecasts before and after this time are distributed differently. The resulting posterior probability mass function is integrated to give a cumulative mass function, which is used to create a weighted median forecast. The effect is to construct an aggregate in which the most heavily weighted forecasts are those which have been made since the probable most recent change in the forecasts' distribution. Kairosis outperforms standard methods, and is especially suitable for geopolitical forecasting tournaments because it is observed to be robust across disparate questions and forecaster distributions.