Bandit-Based Task Assignment for Heterogeneous Crowdsourcing

TL;DR

This paper introduces a contextual bandit approach to optimize task assignment in heterogeneous crowdsourcing, balancing exploration and exploitation to maximize reliable labels within a limited budget.

Contribution

It proposes a novel bandit-based method tailored for heterogeneous crowdsourcing, addressing task diversity and worker reliability, with theoretical regret analysis and experimental validation.

Findings

The method effectively balances exploration and exploitation.

Theoretical regret bounds are established.

Experimental results demonstrate practical effectiveness.

Abstract

We consider a task assignment problem in crowdsourcing, which is aimed at collecting as many reliable labels as possible within a limited budget. A challenge in this scenario is how to cope with the diversity of tasks and the task-dependent reliability of workers, e.g., a worker may be good at recognizing the name of sports teams, but not be familiar with cosmetics brands. We refer to this practical setting as heterogeneous crowdsourcing. In this paper, we propose a contextual bandit formulation for task assignment in heterogeneous crowdsourcing, which is able to deal with the exploration-exploitation trade-off in worker selection. We also theoretically investigate the regret bounds for the proposed method, and demonstrate its practical usefulness experimentally.