From the Lab to People's Home: Lessons from Accessing Blind Participants' Interactions via Smart Glasses in Remote Studies

TL;DR

This study explores using smart glasses for remote usability testing with blind participants, revealing their potential and limitations in capturing interactions and highlighting challenges like hardware support and connectivity.

Contribution

It introduces a novel method of remote observation using smart glasses in blind user studies and shares practical lessons learned from real-world deployment.

Findings

Smart glasses captured 58.7% of interactions via first-person view.

Third-person view captured only 3.7% of interactions.

Challenges include limited screen reader support, battery life, and internet connectivity.

Abstract

Researchers have adopted remote methods, such as online surveys and video conferencing, to overcome challenges in conducting in-person usability testing, such as participation, user representation, and safety. However, remote user evaluation on hardware testbeds is limited, especially for blind participants, as such methods restrict access to observations of user interactions. We employ smart glasses in usability testing with blind people and share our lessons from a case study conducted in blind participants' homes (N=12), where the experimenter can access participants' activities via dual video conferencing: a third-person view via a laptop camera and a first-person view via smart glasses worn by the participant. We show that smart glasses hold potential for observing participants' interactions with smartphone testbeds remotely; on average 58.7% of the interactions were fully captured via the first-person view compared to 3.7% via the third-person. However, this gain is not uniform across participants as it is susceptible to head movements orienting the ear towards a sound source, which highlights the need for a more inclusive camera form factor. We also share our lessons learned when it comes to dealing with lack of screen reader support in smart glasses, a rapidly draining battery, and Internet connectivity in remote studies with blind participants.