Mixed Precision of Quantization of Transformer Language Models for Speech Recognition

TL;DR

This paper introduces novel mixed precision quantization methods for Transformer-based speech recognition models, automatically optimizing local precision to significantly reduce model size without performance loss.

Contribution

It proposes two techniques for automatic mixed precision quantization of Transformers, improving compression and accuracy in speech recognition systems.

Findings

Achieved up to 16x model size reduction without accuracy loss.

Reduced WER by up to 1.7% absolute on speech recognition tasks.

Demonstrated effectiveness on PTB and Switchboard datasets.

Abstract

State-of-the-art neural language models represented by Transformers are becoming increasingly complex and expensive for practical applications. Low-bit deep neural network quantization techniques provides a powerful solution to dramatically reduce their model size. Current low-bit quantization methods are based on uniform precision and fail to account for the varying performance sensitivity at different parts of the system to quantization errors. To this end, novel mixed precision DNN quantization methods are proposed in this paper. The optimal local precision settings are automatically learned using two techniques. The first is based on a quantization sensitivity metric in the form of Hessian trace weighted quantization perturbation. The second is based on mixed precision Transformer architecture search. Alternating direction methods of multipliers (ADMM) are used to efficiently train mixed precision quantized DNN systems. Experiments conducted on Penn Treebank (PTB) and a Switchboard corpus trained LF-MMI TDNN system suggest the proposed mixed precision Transformer quantization techniques achieved model size compression ratios of up to 16 times over the full precision baseline with no recognition performance degradation. When being used to compress a larger full precision Transformer LM with more layers, overall word error rate (WER) reductions up to 1.7% absolute (18% relative) were obtained.