An Incremental Truth Inference Approach to Aggregate Crowdsourcing Contributions in Games with a Purpose

TL;DR

This paper presents an incremental truth inference method tailored for aggregating contributions in Games with a Purpose, demonstrating its advantages through experiments on real GWAP data.

Contribution

It introduces a novel incremental truth inference approach specifically designed for GWAPs, with formalization and experimental validation against existing methods.

Findings

Outperforms state-of-the-art approaches in GWAP data aggregation

Effective in handling contributions from game players

Validated on multiple GWAP datasets

Abstract

We introduce our approach for incremental truth inference over the contributions provided by players of Games with a Purpose: we motivate the need for such a method with the specificity of GWAP vs. traditional crowdsourcing; we explain and formalize the proposed process and we explain its positive consequences; finally, we illustrate the results of an experimental comparison with state-of-the-art approaches, performed on data collected through two different GWAPs, thus showing the properties of our proposed framework.