# Scalable Multi Corpora Neural Language Models for ASR

## TL;DR

This paper introduces scalable neural language models for large-scale ASR, addressing training, latency, and bias challenges, resulting in significant WER reduction in a practical system.

## Contribution

It presents methods for training neural language models on diverse corpora, managing latency, and handling bias, enabling effective deployment in large-scale ASR systems.

## Key findings

- Achieved 6.2% relative WER reduction in second-pass rescoring
- Demonstrated solutions for training on heterogeneous data
- Maintained minimal latency impact

## Abstract

Neural language models (NLM) have been shown to outperform conventional n-gram language models by a substantial margin in Automatic Speech Recognition (ASR) and other tasks. There are, however, a number of challenges that need to be addressed for an NLM to be used in a practical large-scale ASR system. In this paper, we present solutions to some of the challenges, including training NLM from heterogenous corpora, limiting latency impact and handling personalized bias in the second-pass rescorer. Overall, we show that we can achieve a 6.2% relative WER reduction using neural LM in a second-pass n-best rescoring framework with a minimal increase in latency.