Evaluating Singleplayer and Multiplayer in Human Computation Games

TL;DR

This paper investigates how social conditions and scoring mechanics in human computation games influence player experience, accuracy, and completion rates, using a Super Mario Bros.-based game in a human subjects study.

Contribution

It introduces a novel design approach for HCGs and analyzes the effects of social and scoring mechanics on player engagement and performance.

Findings

Multiplayer conditions increased engagement but varied in accuracy.

Collaborative scoring improved task accuracy compared to competitive.

Design tradeoffs between social mechanics and game effectiveness were identified.

Abstract

Human computation games (HCGs) can provide novel solutions to intractable computational problems, help enable scientific breakthroughs, and provide datasets for artificial intelligence. However, our knowledge about how to design and deploy HCGs that appeal to players and solve problems effectively is incomplete. We present an investigatory HCG based on Super Mario Bros. We used this game in a human subjects study to investigate how different social conditions---singleplayer and multiplayer---and scoring mechanics---collaborative and competitive---affect players' subjective experiences, accuracy at the task, and the completion rate. In doing so, we demonstrate a novel design approach for HCGs, and discuss the benefits and tradeoffs of these mechanics in HCG design.