A Survey on Task Assignment in Crowdsourcing

TL;DR

This survey reviews various task assignment methods in crowdsourcing, focusing on their approaches to improve data quality by dynamically adjusting workflows, addressing challenges, and exploring future research directions.

Contribution

It provides a comprehensive overview of task assignment techniques in crowdsourcing, highlighting their differences, challenges, and potential future developments.

Findings

Different methods estimate worker performance variably.

Task assignment strategies address heterogeneous tasks and question allocation.

Future directions include improving adaptability and platform integration.

Abstract

Quality improvement methods are essential to gathering high-quality crowdsourced data, both for research and industry applications. A popular and broadly applicable method is task assignment that dynamically adjusts crowd workflow parameters. In this survey, we review task assignment methods that address: heterogeneous task assignment, question assignment, and plurality problems in crowdsourcing. We discuss and contrast how these methods estimate worker performance, and highlight potential challenges in their implementation. Finally, we discuss future research directions for task assignment methods, and how crowdsourcing platforms and other stakeholders can benefit from them.