From Zero to The Hero: A Collaborative Market Aware Recommendation System for Crowd Workers

TL;DR

This paper introduces a collaborative recommendation system for crowd workers in software crowdsourcing, aiming to improve task success rates by predicting worker success probabilities based on multiple behavioral metrics.

Contribution

It presents a novel recommendation method that leverages collaboration history, preferences, specialty, and proficiency to enhance worker success prediction in crowdsourcing markets.

Findings

Achieved up to 86% success rate using the system

Reduced task dropping rate among crowd workers

Demonstrated effectiveness on 260 active workers

Abstract

The success of software crowdsourcing depends on active and trustworthy pool of worker supply. The uncertainty of crowd workers' behaviors makes it challenging to predict workers' success and plan accordingly. In a competitive crowdsourcing marketplace, competition for success over shared tasks adds another layer of uncertainty in crowd workers' decision-making process. Preliminary analysis on software worker behaviors reveals an alarming task dropping rate of 82.9%. These factors lead to the need for an automated recommendation system for CSD workers to improve the visibility and predictability of their success in the competition. To that end, this paper proposes a collaborative recommendation system for crowd workers. The proposed recommendation system method uses five input metrics based on workers' collaboration history in the pool, workers' preferences in taking tasks in terms of monetary prize and duration, workers' specialty, and workers' proficiency. The proposed method then recommends the most suitable tasks for a worker to compete on based on workers' probability of success in the task. Experimental results on 260 active crowd workers demonstrate that just following the top three success probabilities of task recommendations, workers can achieve success up to 86%