Collective intelligence in science: direct elicitation of diverse information from experts with unknown information structure

TL;DR

This paper introduces a simple, interpretable prediction market mechanism integrated with chat to aggregate diverse private expert information on complex scientific hypotheses, facilitating collaborative analysis without prior knowledge of information structure.

Contribution

It proposes a novel self-resolving prediction market system that encourages experts to share private information directly, enabling effective aggregation even with unknown information structures.

Findings

System reaches equilibrium with experts sharing information

Aggregates diverse private data into interpretable form

Supports large-scale collaborative scientific studies

Abstract

Suppose we need a deep collective analysis of an open scientific problem: there is a complex scientific hypothesis and a large online group of mutually unrelated experts with relevant private information of a diverse and unpredictable nature. This information may be results of experts' individual experiments, original reasoning of some of them, results of AI systems they use, etc. We propose a simple mechanism based on a self-resolving play-money prediction market entangled with a chat. We show that such a system can easily be brought to an equilibrium where participants directly share their private information on the hypothesis through the chat and trade as if the market were resolved in accordance with the truth of the hypothesis. This approach will lead to efficient aggregation of relevant information in a completely interpretable form even if the ground truth cannot be established and experts initially know nothing about each other and cannot perform complex Bayesian calculations. Finally, by rewarding the experts with some real assets proportionally to the play money they end up with, we can get an innovative way to fund large-scale collaborative studies of any type.