Towards Green ASR: Lossless 4-bit Quantization of a Hybrid TDNN System on the 300-hr Switchboard Corpus

TL;DR

This paper introduces a low-footprint, high-performance 4-bit quantized ASR system using neural architectural compression and mixed precision quantization, achieving significant compression without increasing word error rate on the Switchboard corpus.

Contribution

It develops a novel mixed precision quantization approach with neural architectural compression for TDNN and Transformer models, optimizing performance and size.

Findings

Achieved 13.6x lossless compression ratio over full precision systems.

Outperformed uniform precision baseline systems in word error rate.

Maintained statistically insignificant WER increase with aggressive quantization.

Abstract

State of the art time automatic speech recognition (ASR) systems are becoming increasingly complex and expensive for practical applications. This paper presents the development of a high performance and low-footprint 4-bit quantized LF-MMI trained factored time delay neural networks (TDNNs) based ASR system on the 300-hr Switchboard corpus. A key feature of the overall system design is to account for the fine-grained, varying performance sensitivity at different model components to quantization errors. To this end, a set of neural architectural compression and mixed precision quantization approaches were used to facilitate hidden layer level auto-configuration of optimal factored TDNN weight matrix subspace dimensionality and quantization bit-widths. The proposed techniques were also used to produce 2-bit mixed precision quantized Transformer language models. Experiments conducted on the Switchboard data suggest that the proposed neural architectural compression and mixed precision quantization techniques consistently outperform the uniform precision quantised baseline systems of comparable bit-widths in terms of word error rate (WER). An overall "lossless" compression ratio of 13.6 was obtained over the baseline full precision system including both the TDNN and Transformer components while incurring no statistically significant WER increase.