Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios

TL;DR

Flexi-Transducer is a unified on-device speech recognition model that adaptively balances latency and accuracy across multiple domains using domain-specific techniques and a domain indicator, optimizing performance for diverse use-cases.

Contribution

The paper introduces Flexi-Transducer, a novel single model that employs domain-specific segment size adjustments, a restricted RNNT loss, and a domain indicator vector to optimize latency and accuracy for multi-domain on-device ASR.

Findings

Improved WER for dictation scenarios.

Reduced real-time factor for voice commands.

Achieved flexible latency-accuracy trade-offs.

Abstract

Often, the storage and computational constraints of embeddeddevices demand that a single on-device ASR model serve multiple use-cases / domains. In this paper, we propose aFlexibleTransducer(FlexiT) for on-device automatic speech recognition to flexibly deal with multiple use-cases / domains with different accuracy and latency requirements. Specifically, using a single compact model, FlexiT provides a fast response for voice commands, and accurate transcription but with more latency for dictation. In order to achieve flexible and better accuracy and latency trade-offs, the following techniques are used. Firstly, we propose using domain-specific altering of segment size for Emformer encoder that enables FlexiT to achieve flexible de-coding. Secondly, we use Alignment Restricted RNNT loss to achieve flexible fine-grained control on token emission latency for different domains. Finally, we add a domain indicator vector as an additional input to the FlexiT model. Using the combination of techniques, we show that a single model can be used to improve WERs and real time factor for dictation scenarios while maintaining optimal latency for voice commands use-cases