The Surprising Benefits of Base Rate Neglect in Robust Aggregation

TL;DR

This paper shows that in robust forecast aggregation, a moderate level of base rate neglect by experts can reduce worst-case regret, challenging the assumption that Bayesian updating always yields optimal predictions.

Contribution

It introduces a model where experts exhibit partial base rate neglect and demonstrates that intermediate neglect levels can improve aggregation robustness.

Findings

Regret as a function of base rate neglect degree has a V-shape.

Intermediate neglect levels can outperform full Bayesian updating.

Proposes a new aggregator robust to unknown neglect levels.

Abstract

Robust aggregation integrates predictions from multiple experts without knowledge of the experts' information structures. Prior work assumes experts are Bayesian, providing predictions as perfect posteriors based on their signals. However, real-world experts often deviate systematically from Bayesian reasoning. Our work considers experts who tend to ignore the base rate. We find that a certain degree of base rate neglect helps with robust forecast aggregation. Specifically, we consider a forecast aggregation problem with two experts who each predict a binary world state after observing private signals. Unlike previous work, we model experts exhibiting base rate neglect, where they incorporate the base rate information to degree $\lambda\in[0,1]$, with $\lambda=0$ indicating complete ignorance and $\lambda=1$ perfect Bayesian updating. To evaluate aggregators' performance, we adopt Arieli et al. (2018)'s worst-case regret model, which measures the maximum regret across the set of considered information structures compared to an omniscient benchmark. Our results reveal the surprising V-shape of regret as a function of $\lambda$. That is, predictions with an intermediate incorporating degree of base rate $\lambda<1$ can counter-intuitively lead to lower regret than perfect Bayesian posteriors with $\lambda=1$. We additionally propose a new aggregator with low regret robust to unknown $\lambda$. Finally, we conduct an empirical study to test the base rate neglect model and evaluate the performance of various aggregators.