Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision

TL;DR

Mechanical Novel introduces a reflection and revision-based crowdsourcing approach that enables non-expert crowds to produce complex, interdependent creative work like fiction stories by focusing on high-level goals.

Contribution

The paper presents a novel crowdsourcing technique inspired by expert authors, allowing non-experts to collaboratively create complex, interdependent stories through reflection and goal-oriented revision.

Findings

Mechanical Novel produced higher-quality stories than traditional workflows.

High-level goal orientation improves crowd coordination on complex tasks.

Reflection and revision cycles enhance creative output quality.

Abstract

Crowdsourcing systems accomplish large tasks with scale and speed by breaking work down into independent parts. However, many types of complex creative work, such as fiction writing, have remained out of reach for crowds because work is tightly interdependent: changing one part of a story may trigger changes to the overall plot and vice versa. Taking inspiration from how expert authors write, we propose a technique for achieving interdependent complex goals with crowds. With this technique, the crowd loops between reflection, to select a high-level goal, and revision, to decompose that goal into low-level, actionable tasks. We embody this approach in Mechanical Novel, a system that crowdsources short fiction stories on Amazon Mechanical Turk. In a field experiment, Mechanical Novel resulted in higher-quality stories than an iterative crowdsourcing workflow. Our findings suggest that orienting crowd work around high-level goals may enable workers to coordinate their effort to accomplish complex work.