Natural Notation for the Domestic Internet of Things

TL;DR

This paper investigates how natural language and visual cues can be used to enable end-users to program IoT devices in smart homes through reminders and instructions.

Contribution

It introduces a novel approach to natural language instructions for IoT devices, analyzing linguistic and visual strategies used in user-generated reminders.

Findings

People use diverse linguistic strategies for IoT instructions

Graphical resources enhance clarity in user instructions

Design guidelines for IoT end-user programming are proposed

Abstract

This study explores the use of natural language to give instructions that might be interpreted by Internet of Things (IoT) devices in a domestic `smart home' environment. We start from the proposition that reminders can be considered as a type of end-user programming, in which the executed actions might be performed either by an automated agent or by the author of the reminder. We conducted an experiment in which people wrote sticky notes specifying future actions in their home. In different conditions, these notes were addressed to themselves, to others, or to a computer agent.We analyse the linguistic features and strategies that are used to achieve these tasks, including the use of graphical resources as an informal visual language. The findings provide a basis for design guidance related to end-user development for the Internet of Things.