Recruiting Software Engineers on Prolific

TL;DR

This paper discusses the use of Prolific, an academic crowdsourcing platform, for recruiting software engineering research participants, highlighting its advantages over traditional methods and detailing the process of conducting sample studies.

Contribution

It provides an empirical account of recruiting software engineers on Prolific, including study types, selection criteria, and power analysis, demonstrating its effectiveness for research.

Findings

Prolific enables targeted recruitment of software engineers.

The platform allows for efficient data collection with controlled sampling.

Sample studies on Prolific can meet statistical power requirements.

Abstract

Recruiting participants for software engineering research has been a primary concern of the human factors community. This is particularly true for quantitative investigations that require a minimum sample size not to be statistically underpowered. Traditional data collection techniques, such as mailing lists, are highly doubtful due to self-selection biases. The introduction of crowdsourcing platforms allows researchers to select informants with the exact requirements foreseen by the study design, gather data in a concise time frame, compensate their work with fair hourly pay, and most importantly, have a high degree of control over the entire data collection process. This experience report discusses our experience conducting sample studies using Prolific, an academic crowdsourcing platform. Topics discussed are the type of studies, selection processes, and power computation.