A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes

TL;DR

This paper introduces a dynamic cascaded encoder ASR model that unifies multiple model sizes, significantly reducing size and power consumption while maintaining quality, and simplifies deployment across various scenarios.

Contribution

It presents a novel unified model architecture with techniques to optimize performance and efficiency for different deployment sizes, reducing engineering complexity.

Findings

30% smaller model size with 33% less power consumption

Unified model achieves 37% total size reduction

Minimal quality loss across different model sizes

Abstract

In this paper, we propose a dynamic cascaded encoder Automatic Speech Recognition (ASR) model, which unifies models for different deployment scenarios. Moreover, the model can significantly reduce model size and power consumption without loss of quality. Namely, with the dynamic cascaded encoder model, we explore three techniques to maximally boost the performance of each model size: 1) Use separate decoders for each sub-model while sharing the encoders; 2) Use funnel-pooling to improve the encoder efficiency; 3) Balance the size of causal and non-causal encoders to improve quality and fit deployment constraints. Overall, the proposed large-medium model has 30% smaller size and reduces power consumption by 33%, compared to the baseline cascaded encoder model. The triple-size model that unifies the large, medium, and small models achieves 37% total size reduction with minimal quality loss, while substantially reducing the engineering efforts of having separate models.