Combining Probability Forecasts and Understanding Probability Extremizing through Information Diversity

TL;DR

This paper introduces a new framework for combining subjective probability forecasts that accounts for information diversity among forecasters, providing a principled approach to extremizing aggregate predictions based on shared information.

Contribution

It proposes a novel model that describes how information overlap among forecasters influences the optimal extremization of combined probability forecasts.

Findings

The model quantifies how information overlap affects extremizing.

Extremizing is optimal when forecasters have diverse information.

Provides a interpretable framework for forecast aggregation.

Abstract

Randomness in scientific estimation is generally assumed to arise from unmeasured or uncontrolled factors. However, when combining subjective probability estimates, heterogeneity stemming from people's cognitive or information diversity is often more important than measurement noise. This paper presents a novel framework that uses partially overlapping information sources. A specific model is proposed within that framework and applied to the task of aggregating the probabilities given by a group of forecasters who predict whether an event will occur or not. Our model describes the distribution of information across forecasters in terms of easily interpretable parameters and shows how the optimal amount of extremizing of the average probability forecast (shifting it closer to its nearest extreme) varies as a function of the forecasters' information overlap. Our model thus gives a more principled understanding of the historically ad hoc practice of extremizing average forecasts.