"Hey Model!" - Natural User Interactions and Agency in Accessible Interactive 3D Models

TL;DR

This paper explores how blind or low vision users interact with accessible 3D models, revealing preferred multi-modal techniques and the importance of agency to enhance understanding and engagement.

Contribution

It introduces a hierarchy of interaction methods for accessible 3D models and demonstrates their effectiveness through user studies.

Findings

Users prefer tactile exploration first

Touch gestures are used to trigger audio labels

Natural language helps confirm understanding

Abstract

While developments in 3D printing have opened up opportunities for improved access to graphical information for people who are blind or have low vision (BLV), they can provide only limited detailed and contextual information. Interactive 3D printed models (I3Ms) that provide audio labels and/or a conversational agent interface potentially overcome this limitation. We conducted a Wizard-of-Oz exploratory study to uncover the multi-modal interaction techniques that BLV people would like to use when exploring I3Ms, and investigated their attitudes towards different levels of model agency. These findings informed the creation of an I3M prototype of the solar system. A second user study with this model revealed a hierarchy of interaction, with BLV users preferring tactile exploration, followed by touch gestures to trigger audio labels, and then natural language to fill in knowledge gaps and confirm understanding.