On Cost-Effective Incentive Mechanisms in Microtask Crowdsourcing

TL;DR

This paper investigates cost-effective incentive mechanisms in microtask crowdsourcing, proposing a new quality-aware training approach that ensures high-quality solutions at low costs, supported by theoretical analysis and behavioral experiments.

Contribution

It introduces a novel incentive mechanism using quality-aware training to achieve high-quality results with minimal costs in microtask crowdsourcing.

Findings

Proved lower bounds on worker costs for existing mechanisms.

Designed a mechanism that guarantees high quality at arbitrarily low cost.

Validated effectiveness through behavioral experiments.

Abstract

While microtask crowdsourcing provides a new way to solve large volumes of small tasks at a much lower price compared with traditional in-house solutions, it suffers from quality problems due to the lack of incentives. On the other hand, providing incentives for microtask crowdsourcing is challenging since verifying the quality of submitted solutions is so expensive that will negate the advantage of microtask crowdsourcing. We study cost-effective incentive mechanisms for microtask crowdsourcing in this paper. In particular, we consider a model with strategic workers, where the primary objective of a worker is to maximize his own utility. Based on this model, we analyze two basic mechanisms widely adopted in existing microtask crowdsourcing applications and show that, to obtain high quality solutions from workers, their costs are constrained by some lower bounds. We then propose a cost-effective mechanism that employs quality-aware worker training as a tool to stimulate workers to provide high quality solutions. We prove theoretically that the proposed mechanism, when properly designed, can obtain high quality solutions with an arbitrarily low cost. Beyond its theoretical guarantees, we further demonstrate the effectiveness of our proposed mechanisms through a set of behavioral experiments.