Sign Language-Based versus Touch-Based Input for Deaf Users with Interactive Personal Assistants in Simulated Kitchen Environments

TL;DR

This study compares sign language and touch-based input methods for deaf users interacting with smart home assistants in a simulated kitchen, highlighting user preferences and challenges with dirty hands affecting signing.

Contribution

It introduces a comparative evaluation of ASL and touchscreen inputs for IPAs in a realistic scenario, emphasizing usability and user preferences.

Findings

Participants preferred ASL over touch when hands are dirty.

Sign language input received higher enthusiasm and Netpromoter scores.

Dirty hands may influence signing behavior, impacting IPA design.

Abstract

In this study, we assess the usability of interactive personal assistants (IPAs), such as Amazon Alexa, in a simulated kitchen smart home environment, with deaf and hard of hearing users. Participants engage in activities in a way that causes their hands to get dirty. With these dirty hands, they are tasked with two different input methods for IPAs: American Sign Language (ASL) in a Wizard-of-Oz design, and smart home apps with a touchscreen. Usability ratings show that participants significantly preferred ASL over touch-based apps with dirty hands, although not to a larger extent than in comparable previous work with clean hands. Participants also expressed significant enthusiasm for ASL-based IPA interaction in Netpromoter scores and in questions about their overall preferences. Preliminary observations further suggest that having dirty hands may affect the way people sign, which may pose challenges for building IPAs that natively support sign language input.