Reprowd: Crowdsourced Data Processing Made Reproducible

TL;DR

Reprowd is a system designed to enhance reproducibility in crowdsourced data processing research, addressing a gap in cross-disciplinary cooperation and open-sourcing the tool for community use.

Contribution

The paper introduces Reprowd, a novel system that simplifies the reproduction of crowdsourced data processing experiments, promoting transparency and collaboration.

Findings

Reprowd facilitates easy reproduction of crowdsourced data experiments.

Open-sourcing Reprowd encourages community adoption and validation.

Improves cross-disciplinary research collaboration.

Abstract

Crowdsourcing is a multidisciplinary research area including disciplines like artificial intelligence, human-computer interaction, database, and social science. To facilitate cooperation across disciplines, reproducibility is a crucial factor, but unfortunately, it has not gotten enough attention in the HCOMP community. In this paper, we present Reprowd, a system aiming to make it easy to reproduce crowdsourced data processing research. We have open sourced Reprowd at http://sfu-db.github.io/reprowd/.