Error-driven Fixed-Budget ASR Personalization for Accented Speakers

TL;DR

This paper introduces an error-driven sentence selection method for efficient speaker-specific ASR model personalization within fixed data collection budgets, improving recognition accuracy.

Contribution

It proposes a novel approach to select speaker utterances based on phoneme-level error models, enhancing personalization efficiency over random selection.

Findings

Selected sentences have higher error rates than random samples.

Fine-tuning on error-selected sentences yields better WER improvements.

Method enables efficient speaker data collection under budget constraints.

Abstract

We consider the task of personalizing ASR models while being constrained by a fixed budget on recording speaker-specific utterances. Given a speaker and an ASR model, we propose a method of identifying sentences for which the speaker's utterances are likely to be harder for the given ASR model to recognize. We assume a tiny amount of speaker-specific data to learn phoneme-level error models which help us select such sentences. We show that speaker's utterances on the sentences selected using our error model indeed have larger error rates when compared to speaker's utterances on randomly selected sentences. We find that fine-tuning the ASR model on the sentence utterances selected with the help of error models yield higher WER improvements in comparison to fine-tuning on an equal number of randomly selected sentence utterances. Thus, our method provides an efficient way of collecting speaker utterances under budget constraints for personalizing ASR models.