SpeechAgent: An End-to-End Mobile Infrastructure for Speech Impairment Assistance
Haowei Lou, Chengkai Huang, Hye-young Paik, Yongquan Hu, Aaron Quigley, Wen Hu, Lina Yao

TL;DR
SpeechAgent is a mobile infrastructure that combines advanced speech processing and large language models to provide real-time, personalized communication support for individuals with speech impairments, enabling practical daily use.
Contribution
The paper introduces SpeechAgent, a novel mobile system integrating LLM-driven reasoning with speech processing for assistive communication, optimized for real-world deployment.
Findings
Effective real-time speech processing on mobile devices
High accuracy and speech quality in impaired speech datasets
Low latency performance suitable for daily use
Abstract
Speech is essential for human communication, yet millions of people face impairments such as dysarthria, stuttering, and aphasia conditions that often lead to social isolation and reduced participation. Despite recent progress in automatic speech recognition (ASR) and text-to-speech (TTS) technologies, accessible web and mobile infrastructures for users with impaired speech remain limited, hindering the practical adoption of these advances in daily communication. To bridge this gap, we present SpeechAgent, a mobile SpeechAgent designed to facilitate people with speech impairments in everyday communication. The system integrates large language model (LLM)- driven reasoning with advanced speech processing modules, providing adaptive support tailored to diverse impairment types. To ensure real-world practicality, we develop a structured deployment pipeline that enables real-time speech…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpeech Recognition and Synthesis · Stuttering Research and Treatment · Assistive Technology in Communication and Mobility
