Evaluation of a Sign Language Avatar on Comprehensibility, User Experience \& Acceptability

TL;DR

This study evaluates how adjustable sign language avatars on Hololens 2 affect user experience and comprehension, finding limited improvements and emphasizing the need for better animation and interface design.

Contribution

It provides an empirical analysis of adjustable SL avatars' impact on usability, highlighting the importance of default comprehensibility and interface improvements.

Findings

Adjustability preferences did not improve comprehension or UX.

Missing facial expressions and implementation issues hinder effectiveness.

Stress was higher with adjustable avatars, indicating lower usability.

Abstract

This paper presents an investigation into the impact of adding adjustment features to an existing sign language (SL) avatar on a Microsoft Hololens 2 device. Through a detailed analysis of interactions of expert German Sign Language (DGS) users with both adjustable and non-adjustable avatars in a specific use case, this study identifies the key factors influencing the comprehensibility, the user experience (UX), and the acceptability of such a system. Despite user preference for adjustable settings, no significant improvements in UX or comprehensibility were observed, which remained at low levels, amid missing SL elements (mouthings and facial expressions) and implementation issues (indistinct hand shapes, lack of feedback and menu positioning). Hedonic quality was rated higher than pragmatic quality, indicating that users found the system more emotionally or aesthetically pleasing than functionally useful. Stress levels were higher for the adjustable avatar, reflecting lower performance, greater effort and more frustration. Additionally, concerns were raised about whether the Hololens adjustment gestures are intuitive and easy to familiarise oneself with. While acceptability of the concept of adjustability was generally positive, it was strongly dependent on usability and animation quality. This study highlights that personalisation alone is insufficient, and that SL avatars must be comprehensible by default. Key recommendations include enhancing mouthing and facial animation, improving interaction interfaces, and applying participatory design.