Optimal Crowdsourcing Contests

TL;DR

This paper develops a theoretical framework for designing optimal crowdsourcing contests, modeling them as all-pay auctions, and shows they approximate traditional procurement methods within a factor of 2 to 4 depending on distribution regularity.

Contribution

It introduces a theory for optimal crowdsourcing contests based on virtual valuation optimization, extending auction design principles to crowdsourcing scenarios.

Findings

Optimal contests are virtual valuation optimizers.

Crowdsourcing contests are 2-approximate for regular distributions.

They are 4-approximate for non-regular distributions.

Abstract

We study the design and approximation of optimal crowdsourcing contests. Crowdsourcing contests can be modeled as all-pay auctions because entrants must exert effort up-front to enter. Unlike all-pay auctions where a usual design objective would be to maximize revenue, in crowdsourcing contests, the principal only benefits from the submission with the highest quality. We give a theory for optimal crowdsourcing contests that mirrors the theory of optimal auction design: the optimal crowdsourcing contest is a virtual valuation optimizer (the virtual valuation function depends on the distribution of contestant skills and the number of contestants). We also compare crowdsourcing contests with more conventional means of procurement. In this comparison, crowdsourcing contests are relatively disadvantaged because the effort of losing contestants is wasted. Nonetheless, we show that crowdsourcing contests are 2-approximations to conventional methods for a large family of "regular" distributions, and 4-approximations, otherwise.