Task Selection for Bandit-Based Task Assignment in Heterogeneous Crowdsourcing

TL;DR

This paper investigates task selection strategies in bandit-based crowdsourcing, demonstrating that the least confidence approach notably enhances task assignment efficiency compared to other methods.

Contribution

It introduces a comparative analysis of task selection strategies in bandit-based crowdsourcing, highlighting the effectiveness of the least confidence method.

Findings

Least confidence strategy improves task assignment performance

Bandit-based worker selection is effective in crowdsourcing

Experimental comparison of active learning strategies

Abstract

Task selection (picking an appropriate labeling task) and worker selection (assigning the labeling task to a suitable worker) are two major challenges in task assignment for crowdsourcing. Recently, worker selection has been successfully addressed by the bandit-based task assignment (BBTA) method, while task selection has not been thoroughly investigated yet. In this paper, we experimentally compare several task selection strategies borrowed from active learning literature, and show that the least confidence strategy significantly improves the performance of task assignment in crowdsourcing.