Decomposing Crowd Wisdom: Domain-Specific Calibration Dynamics in Prediction Markets

TL;DR

This paper analyzes the calibration of prediction markets, revealing that calibration is a complex, multidimensional phenomenon influenced by domain-specific biases, trade size, and platform microstructure, with implications for interpreting market prices.

Contribution

It decomposes prediction market calibration into multiple components and demonstrates their structure and platform-specific differences using extensive trade data.

Findings

Calibration decomposes into four components explaining 87.3% of variance.

Persistent underconfidence in political markets, with prices biased toward 50%.

Trade size amplifies underconfidence on Kalshi but not on Polymarket.

Abstract

Prediction markets are increasingly used as probability forecasting tools, yet their usefulness depends on calibration, specifically whether a contract trading at 70 cents truly implies a 70% probability. Using 292 million trades across 327,000 binary contracts on Kalshi and Polymarket, this paper shows that calibration is a structured, multidimensional phenomenon. On Kalshi, calibration decomposes into four components (a universal horizon effect, domain-specific biases, domain-by-horizon interactions and a trade-size scale effect) that together explain 87.3% of calibration variance. The dominant pattern is persistent underconfidence in political markets, where prices are chronically compressed toward 50%, and this bias generalises across both exchanges. However, the trade-size scale effect, whereby large trades are associated with amplified underconfidence in politics on Kalshi ($\Delta = 0.53$, 95% confidence interval [0.29, 0.75]), does not replicate on Polymarket ($\Delta = 0.11$, [-0.15, 0.39]), suggesting platform-specific microstructure. A Bayesian hierarchical model confirms the frequentist decomposition with 96.3% posterior predictive coverage. Consumers of prediction market prices who treat them as face-value probabilities will systematically misinterpret them, and the direction of misinterpretation depends on what is being predicted, when and by whom.