Learning of Optimal Forecast Aggregation in Partial Evidence Environments

TL;DR

This paper studies how to learn optimal forecast aggregation in repeated binary event settings with experts sharing partial evidence, showing that under certain conditions, optimal aggregation can be learned efficiently.

Contribution

It introduces a model for Bayesian experts with partial evidence and demonstrates that optimal forecast aggregation can be learned in polynomial time in many cases.

Findings

Optimal aggregation can be learned efficiently in polynomial time.

Characterization of environments where learning is possible or impossible.

Provides algorithms for learning in partial evidence settings.

Abstract

We consider the forecast aggregation problem in repeated settings, where the forecasts are done on a binary event. At each period multiple experts provide forecasts about an event. The goal of the aggregator is to aggregate those forecasts into a subjective accurate forecast. We assume that experts are Bayesian; namely they share a common prior, each expert is exposed to some evidence, and each expert applies Bayes rule to deduce his forecast. The aggregator is ignorant with respect to the information structure (i.e., distribution over evidence) according to which experts make their prediction. The aggregator observes the experts' forecasts only. At the end of each period the actual state is realized. We focus on the question whether the aggregator can learn to aggregate optimally the forecasts of the experts, where the optimal aggregation is the Bayesian aggregation that takes into account all the information (evidence) in the system. We consider the class of partial evidence information structures, where each expert is exposed to a different subset of conditionally independent signals. Our main results are positive; We show that optimal aggregation can be learned in polynomial time in a quite wide range of instances of the partial evidence environments. We provide a tight characterization of the instances where learning is possible and impossible.