VRSL:Exploring the Comprehensibility of 360-Degree Camera Feeds for Sign Language Communication in Virtual Reality

TL;DR

This paper investigates the use of 360-degree camera feeds for sign language communication in VR, assessing comprehensibility, user experience, and technical challenges to improve accessibility for DHH users.

Contribution

It introduces a novel approach to integrating sign language into VR using body-mounted 360-degree cameras and evaluates different mounting positions for effectiveness.

Findings

Shoulder-mounted camera achieved 85% accuracy

Overall comprehension rate was 83.3%

Peripheral distortion affected clarity and user experience

Abstract

This study explores integrating sign language into virtual reality (VR) by examining the comprehensibility and user experience of viewing American Sign Language (ASL) videos captured with body-mounted 360-degree cameras. Ten participants identified ASL signs from videos recorded at three body-mounted positions: head, shoulder, and chest. Results showed the shoulder-mounted camera achieved the highest accuracy (85%), though differences between positions were not statistically significant. Participants noted that peripheral distortion in 360-degree videos impacted clarity, highlighting areas for improvement. Despite challenges, the overall comprehension success rate of 83.3% demonstrates the potential of video-based ASL communication in VR. Feedback emphasized the need to refine camera angles, reduce distortion, and explore alternative mounting positions. Participants expressed a preference for signing over text-based communication in VR, highlighting the importance of developing this approach to enhance accessibility and collaboration for Deaf and Hard of Hearing (DHH) users in virtual environments.