Peer Truth Serum: Incentives for Crowdsourcing Measurements and Opinions

TL;DR

The paper introduces Peer Truth Serum, a novel incentive mechanism designed to motivate truthful and accurate reporting in crowdsourcing tasks involving self-interested agents, ensuring high-quality data collection.

Contribution

It presents the Peer Truth Serum mechanism, the first to guarantee truthful reporting in crowdsourcing with self-interested agents under specific conditions.

Findings

Peer Truth Serum incentivizes truthful reporting.

The mechanism is unique in satisfying key desirable properties.

It effectively discourages random or low-effort reports.

Abstract

Modern decision making tools are based on statistical analysis of abundant data, which is often collected by querying multiple individuals. We consider data collection through crowdsourcing, where independent and self-interested agents, non-experts, report measurements, such as sensor readings, opinions, such as product reviews, or answers to human intelligence tasks. Since the accuracy of information is positively correlated with the effort invested in obtaining it, self-interested agents tend to report low-quality data. Therefore, there is a need for incentives that cover the cost of effort, while discouraging random reports. We propose a novel incentive mechanism called Peer Truth Serum that encourages truthful and accurate reporting, showing that it is the unique mechanism to satisfy a combination of desirable properties.