Deep Bayesian Trust : A Dominant and Fair Incentive Mechanism for Crowd

TL;DR

This paper introduces a novel incentive mechanism for crowdsourcing that combines limited gold standard tasks with peer-based evaluation, ensuring fairness and dominant incentive compatibility at large scale.

Contribution

It proposes a new mechanism that assigns gold tasks selectively and uses transitivity to infer peer accuracy, improving efficiency and fairness over traditional methods.

Findings

Ensures dominant incentive compatibility and fairness.

Reduces the need for gold standard tasks at scale.

Leverages transitivity to infer worker accuracy.

Abstract

An important class of game-theoretic incentive mechanisms for eliciting effort from a crowd are the peer based mechanisms, in which workers are paid by matching their answers with one another. The other classic mechanism is to have the workers solve some gold standard tasks and pay them according to their accuracy on gold tasks. This mechanism ensures stronger incentive compatibility than the peer based mechanisms but assigning gold tasks to all workers becomes inefficient at large scale. We propose a novel mechanism that assigns gold tasks to only a few workers and exploits transitivity to derive accuracy of the rest of the workers from their peers' accuracy. We show that the resulting mechanism ensures a dominant notion of incentive compatibility and fairness.