ILASR: Privacy-Preserving Incremental Learning for Automatic Speech Recognition at Production Scale

TL;DR

This paper presents ILASR, a privacy-preserving incremental learning system for end-to-end speech recognition that improves model performance over time without human annotations, suitable for production environments.

Contribution

It introduces a cloud-based framework for privacy-preserving incremental ASR learning, demonstrating significant model improvements with weak supervision and large batch sizes.

Findings

Achieved 3% improvement over six months without human labels

Enhanced recognition of new words and phrases by 20%

Validated effectiveness of large batch sizes and teacher models

Abstract

Incremental learning is one paradigm to enable model building and updating at scale with streaming data. For end-to-end automatic speech recognition (ASR) tasks, the absence of human annotated labels along with the need for privacy preserving policies for model building makes it a daunting challenge. Motivated by these challenges, in this paper we use a cloud based framework for production systems to demonstrate insights from privacy preserving incremental learning for automatic speech recognition (ILASR). By privacy preserving, we mean, usage of ephemeral data which are not human annotated. This system is a step forward for production levelASR models for incremental/continual learning that offers near real-time test-bed for experimentation in the cloud for end-to-end ASR, while adhering to privacy-preserving policies. We show that the proposed system can improve the production models significantly(3%) over a new time period of six months even in the absence of human annotated labels with varying levels of weak supervision and large batch sizes in incremental learning. This improvement is 20% over test sets with new words and phrases in the new time period. We demonstrate the effectiveness of model building in a privacy-preserving incremental fashion for ASR while further exploring the utility of having an effective teacher model and use of large batch sizes.