Hedging and Approximate Truthfulness in Traditional Forecasting Competitions

TL;DR

This paper formally analyzes traditional forecasting competitions, revealing that contestants may hedge rather than report truthful beliefs, but approximate truthfulness can occur under certain uncertainty conditions.

Contribution

First formal analysis of the traditional forecasting mechanism, showing its incentive issues and conditions for approximate truthfulness.

Findings

Best forecasters may hedge to increase win probability.

Two contestants tend to be approximately truthful under high uncertainty.

Traditional mechanism can incentivize strategic reporting regardless of event number.

Abstract

In forecasting competitions, the traditional mechanism scores the predictions of each contestant against the outcome of each event, and the contestant with the highest total score wins. While it is well-known that this traditional mechanism can suffer from incentive issues, it is folklore that contestants will still be roughly truthful as the number of events grows. Yet thus far the literature lacks a formal analysis of this traditional mechanism. This paper gives the first such analysis. We first demonstrate that the ''long-run truthfulness'' folklore is false: even for arbitrary numbers of events, the best forecaster can have an incentive to hedge, reporting more moderate beliefs to increase their win probability. On the positive side, however, we show that two contestants will be approximately truthful when they have sufficient uncertainty over the relative quality of their opponent and the outcomes of the events, a case which may arise in practice.