Tailored Real-time AR Captioning Interface for Enhancing Learning Experience of Deaf and Hard-of-Hearing (DHH) Students

TL;DR

This paper presents a user-centered designed real-time AR captioning system tailored for DHH students, significantly improving their educational experience and engagement in classroom settings.

Contribution

It introduces a novel AR captioning interface developed through collaborative design with DHH users, demonstrating its effectiveness in enhancing learning experiences.

Findings

87.5% of users preferred the system

Improved engagement and comprehension reported

Effective integration into classroom settings

Abstract

Deaf and hard-of-hearing (DHH) students face significant challenges in specialized educational settings, such as limited exposure to written and spoken language, a lack of tailored educational tools, and restricted access to resources, impacting their language literacy development and overall educational experience. We, therefore, employed a User-Centered Design (UCD) process, collaborating with 8 DHH students and 2 Teachers of the Deaf (ToDs) from a School of Deaf to effectively develop and utilize a real-time captioning augmented reality (AR) system to their school settings, aiming to enhance their learning experience. User study with 24 DHH participants revealed a strong preference (87.5\%) for our system, underscoring its potential to enhance learning experience. We present a comprehensive needs analysis, the UCD process, system implementation, and user feedback, showcasing the effectiveness of tailored AR caption interfaces for DHH students. We also discuss the implications for future development of educational technologies for DHH students.