Bridging ASR and LLMs for Dysarthric Speech Recognition: Benchmarking Self-Supervised and Generative Approaches

TL;DR

This paper benchmarks self-supervised ASR models for dysarthric speech, introduces LLM-based decoding to enhance recognition, and analyzes their generalization and error patterns across severity levels.

Contribution

It systematically evaluates ASR architectures for dysarthric speech and introduces LLM-based decoding to improve accuracy and intelligibility.

Findings

LLM-enhanced decoding improves recognition accuracy.

LLMs help restore phonemes and correct grammar.

Models show varying generalization across datasets.

Abstract

Speech Recognition (ASR) due to phoneme distortions and high variability. While self-supervised ASR models like Wav2Vec, HuBERT, and Whisper have shown promise, their effectiveness in dysarthric speech remains unclear. This study systematically benchmarks these models with different decoding strategies, including CTC, seq2seq, and LLM-enhanced decoding (BART,GPT-2, Vicuna). Our contributions include (1) benchmarking ASR architectures for dysarthric speech, (2) introducing LLM-based decoding to improve intelligibility, (3) analyzing generalization across datasets, and (4) providing insights into recognition errors across severity levels. Findings highlight that LLM-enhanced decoding improves dysarthric ASR by leveraging linguistic constraints for phoneme restoration and grammatical correction.