On the efficacy of the wisdom of crowds to forecast economic indicators

TL;DR

This paper evaluates the effectiveness of aggregating expert forecasts using median and mean in predicting economic indicators, finding median aggregation slightly more effective and highlighting the role of selective attention in forecast accuracy.

Contribution

It provides empirical evidence comparing median and mean aggregation methods in expert forecasts and challenges the assumption that crowds are inherently more accurate than individual experts.

Findings

Median aggregation has a higher probability of beating all participants than mean.

Both median and mean aggregations outperform individual forecasts but are less accurate than winners.

Selective attention likely explains the high accuracy of crowds in forecasting.

Abstract

The interest in the wisdom of crowds stems mainly from the possibility of combining independent forecasts from experts in the hope that many expert minds are better than a few. Hence the relevant subject of study nowadays is the Vox Expertorum rather than Galton's original Vox Populi. Here we use the Federal Reserve Bank of Philadelphia's Survey of Professional Forecasters to analyze $15455$ forecasting contests to predict a variety of economic indicators. We find that the median has advantages over the mean as a method to combine the experts' estimates: the odds that the crowd beats all participants of a forecasting contest is $0.015$ when the aggregation is given by the mean and $0.026$ when it is given by the median. In addition, the median is always guaranteed to beat the majority of the participants, whereas the mean beats that majority in 67 percent of the forecasts only. Both aggregation methods yield a $20$ percent error on the average, which must be contrasted with the $15$ percent error of the contests' winners. A standard time series forecasting algorithm, the ARIMA model, yields a $31$ percent error on the average. However, since the expected error of a randomly selected forecaster is about $22$ percent, our conclusion is that selective attention is the most likely explanation for the mysterious high accuracy of the crowd reported in the literature.