Screen or No Screen? Lessons Learnt from a Real-World Deployment Study of Using Voice Assistants With and Without Touchscreen for Older Adults

TL;DR

This study investigates how older adults interact with voice assistants with and without touchscreens over 40 days, revealing preferences and design insights to improve senior-friendly multimodal interfaces.

Contribution

It provides empirical insights into older adults' interactions with touchscreen-enabled voice assistants in real-world settings, highlighting design implications for future development.

Findings

Participants appreciated visual outputs but preferred speech responses.

Touchscreens did not significantly change interaction preferences.

Six design implications for senior-friendly voice assistants were identified.

Abstract

While voice user interfaces offer increased accessibility due to hands-free and eyes-free interactions, older adults often have challenges such as constructing structured requests and perceiving how such devices operate. Voice-first user interfaces have the potential to address these challenges by enabling multimodal interactions. Standalone voice + touchscreen Voice Assistants (VAs), such as Echo Show, are specific types of devices that adopt such interfaces and are gaining popularity. However, the affordances of the additional touchscreen for older adults are unknown. Through a 40-day real-world deployment with older adults living independently, we present a within-subjects study (N = 16; age M = 82.5, SD = 7.77, min. = 70, max. = 97) to understand how a built-in touchscreen might benefit older adults during device setup, conducting self-report diary survey, and general uses. We found that while participants appreciated the visual outputs, they still preferred to respond via speech instead of touch. We identified six design implications that can inform future innovations of senior-friendly VAs for managing healthcare and improving quality of life.