Enhancing AAC Software for Dysarthric Speakers in e-Health Settings: An Evaluation Using TORGO

TL;DR

This paper evaluates and improves automatic speech recognition for dysarthric speakers in healthcare, addressing dataset biases and leveraging advanced models to enhance communication tools for individuals with speech impairments.

Contribution

It introduces an algorithm to eliminate prompt-overlap in the TORGO dataset and assesses the impact of language models and LLM-based error correction on ASR performance for dysarthric speech.

Findings

Prompt-overlap significantly affects ASR evaluation.

State-of-the-art ASR models perform poorly on dysarthric speech.

LLM-based error correction improves recognition accuracy.

Abstract

Individuals with cerebral palsy (CP) and amyotrophic lateral sclerosis (ALS) frequently face challenges with articulation, leading to dysarthria and resulting in atypical speech patterns. In healthcare settings, communication breakdowns reduce the quality of care. While building an augmentative and alternative communication (AAC) tool to enable fluid communication we found that state-of-the-art (SOTA) automatic speech recognition (ASR) technology like Whisper and Wav2vec2.0 marginalizes atypical speakers largely due to the lack of training data. Our work looks to leverage SOTA ASR followed by domain specific error-correction. English dysarthric ASR performance is often evaluated on the TORGO dataset. Prompt-overlap is a well-known issue with this dataset where phrases overlap between training and test speakers. Our work proposes an algorithm to break this prompt-overlap. After reducing prompt-overlap, results with SOTA ASR models produce extremely high word error rates for speakers with mild and severe dysarthria. Furthermore, to improve ASR, our work looks at the impact of n-gram language models and large-language model (LLM) based multi-modal generative error-correction algorithms like Whispering-LLaMA for a second pass ASR. Our work highlights how much more needs to be done to improve ASR for atypical speakers to enable equitable healthcare access both in-person and in e-health settings.