# Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition

## TL;DR

This paper investigates integrating domain-adapted self-supervised learning models, specifically wav2vec2.0, into ASR systems to improve recognition accuracy for dysarthric and elderly speech, achieving significant WER reductions.

## Contribution

It introduces novel methods for combining wav2vec2.0 with traditional ASR systems and applies multi-modal approaches, resulting in state-of-the-art performance on dysarthric and elderly speech datasets.

## Key findings

- Significant WER reductions of 8.22% and 3.43% absolute on two datasets.
- Achieved lowest published WERs of 22.56% and 18.17%.
- Enhanced recognition of very low intelligibility and unseen words.

## Abstract

Automatic recognition of disordered and elderly speech remains a highly challenging task to date due to the difficulty in collecting such data in large quantities. This paper explores a series of approaches to integrate domain adapted SSL pre-trained models into TDNN and Conformer ASR systems for dysarthric and elderly speech recognition: a) input feature fusion between standard acoustic frontends and domain adapted wav2vec2.0 speech representations; b) frame-level joint decoding of TDNN systems separately trained using standard acoustic features alone and with additional wav2vec2.0 features; and c) multi-pass decoding involving the TDNN/Conformer system outputs to be rescored using domain adapted wav2vec2.0 models. In addition, domain adapted wav2vec2.0 representations are utilized in acoustic-to-articulatory (A2A) inversion to construct multi-modal dysarthric and elderly speech recognition systems. Experiments conducted on the UASpeech dysarthric and DementiaBank Pitt elderly speech corpora suggest TDNN and Conformer ASR systems integrated domain adapted wav2vec2.0 models consistently outperform the standalone wav2vec2.0 models by statistically significant WER reductions of 8.22% and 3.43% absolute (26.71% and 15.88% relative) on the two tasks respectively. The lowest published WERs of 22.56% (52.53% on very low intelligibility, 39.09% on unseen words) and 18.17% are obtained on the UASpeech test set of 16 dysarthric speakers, and the DementiaBank Pitt test set respectively.