Scaling A Simple Approach to Zero-Shot Speech Recognition

TL;DR

This paper introduces MMS Zero-shot, a simple and scalable zero-shot speech recognition method that leverages extensive multilingual training and romanization, significantly improving accuracy on unseen languages without labeled data.

Contribution

The paper presents MMS Zero-shot, a novel approach trained on over a thousand languages, achieving substantial error rate reductions and eliminating the need for labeled data in unseen languages.

Findings

46% relative reduction in character error rate over 100 unseen languages

Error rate only 2.5 times higher than supervised baselines on in-domain data

Uses no labeled data for evaluation languages

Abstract

Despite rapid progress in increasing the language coverage of automatic speech recognition, the field is still far from covering all languages with a known writing script. Recent work showed promising results with a zero-shot approach requiring only a small amount of text data, however, accuracy heavily depends on the quality of the used phonemizer which is often weak for unseen languages. In this paper, we present MMS Zero-shot a conceptually simpler approach based on romanization and an acoustic model trained on data in 1,078 different languages or three orders of magnitude more than prior art. MMS Zero-shot reduces the average character error rate by a relative 46% over 100 unseen languages compared to the best previous work. Moreover, the error rate of our approach is only 2.5x higher compared to in-domain supervised baselines, while our approach uses no labeled data for the evaluation languages at all.