Massive End-to-end Models for Short Search Queries

TL;DR

This paper evaluates large-scale end-to-end speech recognition models, specifically CTC and RNN-T, for offline voice search queries, highlighting the superior performance of RNN-T with larger models and the impact of various training strategies.

Contribution

It provides a comprehensive comparison of CTC and RNN-T models at large scale for voice search, demonstrating RNN-T's advantages in accuracy and robustness to time reduction.

Findings

RNN-T outperforms CTC at similar sizes in WER.

Larger models improve recognition accuracy.

Shallow fusion with language models reduces WER gap.

Abstract

In this work, we investigate two popular end-to-end automatic speech recognition (ASR) models, namely Connectionist Temporal Classification (CTC) and RNN-Transducer (RNN-T), for offline recognition of voice search queries, with up to 2B model parameters. The encoders of our models use the neural architecture of Google's universal speech model (USM), with additional funnel pooling layers to significantly reduce the frame rate and speed up training and inference. We perform extensive studies on vocabulary size, time reduction strategy, and its generalization performance on long-form test sets. Despite the speculation that, as the model size increases, CTC can be as good as RNN-T which builds label dependency into the prediction, we observe that a 900M RNN-T clearly outperforms a 1.8B CTC and is more tolerant to severe time reduction, although the WER gap can be largely removed by LM shallow fusion.