Truthful Information Elicitation from Hybrid Crowds

TL;DR

This paper introduces a novel framework and mechanisms for incentivizing truthful information reporting from heterogeneous crowds, addressing the challenge of diverse agent types without prior knowledge of their distribution.

Contribution

It proposes the first framework for hybrid crowds and two mechanisms—linear transformation and mutual information-based—to ensure truthful reporting and learn agent expertise.

Findings

Mechanisms effectively incentivize truthful reporting from diverse agents.

The approach improves information quality and enables expertise estimation.

Framework addresses heterogeneity without prior agent type knowledge.

Abstract

Suppose a decision maker wants to predict weather tomorrow by eliciting and aggregating information from crowd. How can the decision maker incentivize the crowds to report their information truthfully? Many truthful peer prediction mechanisms have been proposed for homogeneous agents, whose types are drawn from the same distribution. However, in many situations, the population is a hybrid crowd of different types of agents with different forms of information, and the decision maker has neither the identity of any individual nor the proportion of each types of agents in the crowd. Ignoring the heterogeneity among the agent may lead to inefficient of biased information, which would in turn lead to suboptimal decisions. In this paper, we propose the first framework for information elicitation from hybrid crowds, and two mechanisms to motivate agents to report their information truthfully. The first mechanism combines two mechanisms via linear transformations and the second is based on mutual information. With two mechanisms, the decision maker can collect high quality information from hybrid crowds, and learns the expertise of agents.