Neutral Pivoting: Strong Bias Correction for Shared Information

TL;DR

This paper introduces the neutral pivot, a new bias correction method for aggregating crowd predictions that outperforms existing approaches by effectively reducing bias without requiring detailed crowd information.

Contribution

The paper proposes the neutral pivot, a novel bias correction technique that improves upon minimal pivot by achieving larger bias correction while maintaining low error, without estimating crowd composition.

Findings

Neutral pivot outperforms existing methods in real data tests.

It achieves the largest bias correction among similar approaches.

Maintains smaller expected squared error than simple mean.

Abstract

In the absence of historical data for use as forecasting inputs, decision makers often ask a panel of judges to predict the outcome of interest, leveraging the wisdom of the crowd (Surowiecki 2005). Even if the crowd is large and skilled, shared information can bias the simple mean of judges' estimates. Addressing the issue of bias, Palley and Soll (2019) introduces a novel approach called pivoting. Pivoting can take several forms, most notably the powerful and reliable minimal pivot. We build on the intuition of the minimal pivot and propose a more aggressive bias correction known as the neutral pivot. The neutral pivot achieves the largest bias correction of its class that both avoids the need to directly estimate crowd composition or skill and maintains a smaller expected squared error than the simple mean for all considered settings. Empirical assessments on real datasets confirm the effectiveness of the neutral pivot compared to current methods.