InstructableCrowd: Creating IF-THEN Rules for Smartphones via Conversations with the Crowd

TL;DR

InstructableCrowd enables users to program smartphones through conversational interactions with crowd workers, allowing non-programmers to create complex, sensor-based IF-THEN rules for device automation.

Contribution

This paper introduces a crowd-powered conversational system for programming smartphones with multi-part IF-THEN rules, enhancing personalization and complexity beyond simple voice commands.

Findings

Non-programmers can create high-quality IF-THEN rules via conversation.

The system's rules are comparable in quality to manually created rules.

Crowd collaboration enables complex device programming through natural language.

Abstract

Natural language interfaces have become a common part of modern digital life. Chatbots utilize text-based conversations to communicate with users; personal assistants on smartphones such as Google Assistant take direct speech commands from their users; and speech-controlled devices such as Amazon Echo use voice as their only input mode. In this paper, we introduce InstructableCrowd, a crowd-powered system that allows users to program their devices via conversation. The user verbally expresses a problem to the system, in which a group of crowd workers collectively respond and program relevant multi-part IF-THEN rules to help the user. The IF-THEN rules generated by InstructableCrowd connect relevant sensor combinations (e.g., location, weather, device acceleration, etc.) to useful effectors (e.g., text messages, device alarms, etc.). Our study showed that non-programmers can use the conversational interface of InstructableCrowd to create IF-THEN rules that have similar quality compared with the rules created manually. InstructableCrowd generally illustrates how users may converse with their devices, not only to trigger simple voice commands, but also to personalize their increasingly powerful and complicated devices.