Multi-team Formation System for Collaborative Crowdsourcing

TL;DR

This paper introduces a heuristic algorithm for forming optimal multi-team groups in crowdsourcing by considering worker compatibility, expertise, and budget constraints, utilizing social network data and simulation-based analysis.

Contribution

It presents a novel social network-based heuristic optimization method for multi-team formation in crowdsourcing, outperforming traditional hill-climbing approaches.

Findings

Algorithm outperforms hill-climbing in most conditions

Optimal parameters include a temperature decrease rate of about 0.9

Low team size, social network degree, and budget lead to poor evaluation values

Abstract

For complex crowdsourcing tasks that require collaboration between multiple individuals, teams should be formed by considering both worker compatibility and expertise. Furthermore, the nature of crowdsourcing dictates the budget for tasks and workers' remuneration, and excessively large team sizes may reduce collaborative performance. To address these challenges, we propose a heuristic optimization algorithm that leverages social network information to simultaneously form teams with optimized worker compatibility for multiple tasks. In our approach, historical collaboration is represented as a social network, where the edge weights correspond to explicit ratings of worker compatibility. In a simulation experiment using synthetic data, we applied Gaussian process regression to examine the relationship between eight experimental parameters and evaluation values, thereby analyzing the output of the proposed algorithm. To generate the necessary data for regression, we ran the proposed algorithm with experimental parameters that were sequentially estimated using Bayesian optimization. Our experiments revealed that the evaluation values were extremely low when the team size limit, the degree mean of the social network, and the task budget were set to low values. The results also indicate that the proposed algorithm outperformed the hill-climbing method under almost all experimental conditions. In addition, the highest evaluation values were achieved when the simulated annealing temperature decrease rate was approximately 0.9, while smoothing the objective function proved ineffective.