See-Through Captions: Real-Time Captioning on Transparent Display for Deaf and Hard-of-Hearing People
Kenta Yamamoto, Ippei Suzuki, Akihisa Shitara, Yoichi Ochiai

TL;DR
This paper introduces a real-time captioning system for deaf and hard-of-hearing individuals that uses transparent displays to show captions on both sides, enabling comfortable communication without obstructing nonverbal cues.
Contribution
It proposes a novel transparent display-based captioning system that enhances communication quality by addressing recognition errors and preserving nonverbal communication cues.
Findings
Captions are visible on both sides of the transparent display.
The system allows users to see both captions and body language simultaneously.
It improves communication effectiveness for DHH people in real-time.
Abstract
Real-time captioning is a useful technique for deaf and hard-of-hearing (DHH) people to talk to hearing people. With the improvement in device performance and the accuracy of automatic speech recognition (ASR), real-time captioning is becoming an important tool for helping DHH people in their daily lives. To realize higher-quality communication and overcome the limitations of mobile and augmented-reality devices, real-time captioning that can be used comfortably while maintaining nonverbal communication and preventing incorrect recognition is required. Therefore, we propose a real-time captioning system that uses a transparent display. In this system, the captions are presented on both sides of the display to address the problem of incorrect ASR, and the highly transparent display makes it possible to see both the body language and the captions.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTactile and Sensory Interactions · Hand Gesture Recognition Systems · Indoor and Outdoor Localization Technologies
