Accessible Nonverbal Cues to Support Conversations in VR for Blind and Low Vision People

TL;DR

This study introduces audio and haptic cues for nonverbal communication in social VR, significantly improving accessibility for blind and low vision users during conversations.

Contribution

It presents the design and evaluation of accessible nonverbal cues in VR, demonstrating their effectiveness and user preferences for blind and low vision individuals.

Findings

Cues improved detection accuracy of attention in VR conversations.

Participants reported higher confidence with cues.

Users had diverse preferences and uses for the cues.

Abstract

Social VR has increased in popularity due to its affordances for rich, embodied, and nonverbal communication. However, nonverbal communication remains inaccessible for blind and low vision people in social VR. We designed accessible cues with audio and haptics to represent three nonverbal behaviors: eye contact, head shaking, and head nodding. We evaluated these cues in real-time conversation tasks where 16 blind and low vision participants conversed with two other users in VR. We found that the cues were effective in supporting conversations in VR. Participants had statistically significantly higher scores for accuracy and confidence in detecting attention during conversations with the cues than without. We also found that participants had a range of preferences and uses for the cues, such as learning social norms. We present design implications for handling additional cues in the future, such as the challenges of incorporating AI. Through this work, we take a step towards making interpersonal embodied interactions in VR fully accessible for blind and low vision people.