EchoAid: Enhancing Livestream Shopping Accessibility for the DHH Community

TL;DR

EchoAid is a mobile app that leverages speech-to-text, RSVP, and LLMs to improve livestream shopping accessibility for the DHH community, reducing cognitive overload and enhancing user engagement.

Contribution

This work introduces EchoAid, a novel accessibility tool combining speech recognition and LLMs specifically designed for DHH livestream shoppers.

Findings

EchoAid improves information extraction for DHH users.

The app reduces cognitive overload during livestream shopping.

Participants reported more engaging and personalized experiences.

Abstract

Livestream shopping platforms often overlook the accessibility needs of the Deaf and Hard of Hearing (DHH) community, leading to barriers such as information inaccessibility and overload. To tackle these challenges, we developed \textit{EchoAid}, a mobile app designed to improve the livestream shopping experience for DHH users. \textit{EchoAid} utilizes advanced speech-to-text conversion, Rapid Serial Visual Presentation (RSVP) technology, and Large Language Models (LLMs) to simplify the complex information flow in live sales environments. We conducted exploratory studies with eight DHH individuals to identify design needs and iteratively developed the \textit{EchoAid} prototype based on feedback from three participants. We then evaluate the performance of this system in a user study workshop involving 38 DHH participants. Our findings demonstrate the successful design and validation process of \textit{EchoAid}, highlighting its potential to enhance product information extraction, leading to reduced cognitive overload and more engaging and customized shopping experiences for DHH users.