Scaling ASR Improves Zero and Few Shot Learning

TL;DR

This paper demonstrates that scaling up automatic speech recognition models to billions of parameters, combined with data selection and optimization techniques, significantly enhances zero and few-shot learning across diverse domains and styles.

Contribution

The authors introduce data selection and optimization methods to efficiently scale ASR models up to 10 billion parameters, achieving state-of-the-art zero and few-shot performance.

Findings

Models outperform previous benchmarks in zero and few-shot tasks.

Significant improvements in speech recognition for speakers with disorders.

Achieves high performance with substantially less in-domain data.

Abstract

With 4.5 million hours of English speech from 10 different sources across 120 countries and models of up to 10 billion parameters, we explore the frontiers of scale for automatic speech recognition. We propose data selection techniques to efficiently scale training data to find the most valuable samples in massive datasets. To efficiently scale model sizes, we leverage various optimizations such as sparse transducer loss and model sharding. By training 1-10B parameter universal English ASR models, we push the limits of speech recognition performance across many domains. Furthermore, our models learn powerful speech representations with zero and few-shot capabilities on novel domains and styles of speech, exceeding previous results across multiple in-house and public benchmarks. For speakers with disorders due to brain damage, our best zero-shot and few-shot models achieve 22% and 60% relative improvement on the AphasiaBank test set, respectively, while realizing the best performance on public social media videos. Furthermore, the same universal model reaches equivalent performance with 500x less in-domain data on the SPGISpeech financial-domain dataset.