A light-weight and efficient punctuation and word casing prediction model for on-device streaming ASR

TL;DR

This paper introduces a lightweight CNN-BiLSTM model for real-time punctuation and casing prediction in on-device streaming ASR, achieving high accuracy with minimal size and fast inference.

Contribution

It presents a novel, efficient model that outperforms non-Transformer models and rivals Transformer models in size and speed for on-device ASR punctuation and casing prediction.

Findings

9% relative improvement in F1-score over non-Transformer models

Model is 40 times smaller than Transformer models

Inference is 2.5 times faster than Transformer-based approaches

Abstract

Punctuation and word casing prediction are necessary for automatic speech recognition (ASR). With the popularity of on-device end-to-end streaming ASR systems, the on-device punctuation and word casing prediction become a necessity while we found little discussion on this. With the emergence of Transformer, Transformer based models have been explored for this scenario. However, Transformer based models are too large for on-device ASR systems. In this paper, we propose a light-weight and efficient model that jointly predicts punctuation and word casing in real time. The model is based on Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BiLSTM). Experimental results on the IWSLT2011 test set show that the proposed model obtains 9% relative improvement compared to the best of non-Transformer models on overall F1-score. Compared to the representative of Transformer based models, the proposed model achieves comparable results to the representative model while being only one-fortieth its size and 2.5 times faster in terms of inference time. It is suitable for on-device streaming ASR systems. Our code is publicly available.