Studying Breakdowns in Interactions with Smart Speakers

TL;DR

This paper investigates user challenges with smart speakers, focusing on issues of intelligibility and control, and discusses future directions for improving their interaction and interpretability in home IoT environments.

Contribution

It presents new insights into user experiences with smart speakers, highlighting challenges and proposing future research directions for enhancing their physical and contextual intelligibility.

Findings

Users face control and understanding challenges with smart speakers.

Smart speakers' interaction with IoT devices can be opaque.

Future designs should improve physical and situational intelligibility.

Abstract

The popularity of voice-controlled smart speakers with intelligent personal assistants (IPAs) like the Amazon Echo and their increasing use as an interface for other Internet of Things (IoT) technologies in the home provides opportunities to study smart speakers as an emerging and ubiquitous IoT device. Prior research has studied how smart speaker usage has unfolded in homes and how the devices have been integrated into people's daily routines. In this context, researchers have also observed instances of smart speakers' 'black box' behaviour. In this position paper, we present findings from a study we conducted to specifically investigate such challenges people experience around intelligibility and control of their smart speakers, for instance, when the smart speaker interfaces with other IoT systems. Reflecting on our findings, we discuss new possible directions for smart speakers including physical intelligibility, situational physical interaction, and providing access to alternative interpretations in shared and non-shared contexts.