Creating Interactive Behaviors in Early Sketch by Recording and Remixing Crowd Demonstrations

TL;DR

This paper introduces a crowd-powered system that enables early GUI sketches to be augmented with interactive behaviors through demonstration and remixing, enhancing rapid prototyping of interactive interfaces.

Contribution

It extends the Apparition tool to allow crowdsourcing of interactive behavior demonstrations and remixing, improving early-stage GUI prototyping.

Findings

Crowd demonstrations improve behavior fidelity.

Remixing behaviors accelerates future prototyping.

System supports collaborative, real-time GUI development.

Abstract

In the early stages of designing graphical user interfaces (GUIs), the look (appearance) can be easily presented by sketching, but the feel (interactive behaviors) cannot, and often requires an accompanying description of how it works (Myers et al. 2008). We propose to use crowdsourcing to augment early sketches with interactive behaviors generated, used, and reused by collective "wizards-of-oz" as opposed to a single wizard as in prior work (Davis et al. 2007). This demo presents an extension of Apparition (Lasecki et al. 2015), a crowd-powered prototyping tool that allows end users to create functional GUIs using speech and sketch. In Apparition, crowd workers collaborate in real-time on a shared canvas to refine the user-requested sketch interactively, and with the assistance of the end users. Our demo extends this functionality to let crowd workers "demonstrate" the canvas changes that are needed for a behavior and refine their demonstrations to improve the fidelity of interactive behaviors. The system then lets workers "remix" these behaviors to make creating future behaviors more efficient.