EasyQuant: Post-training Quantization via Scale Optimization

TL;DR

EasyQuant is a post-training quantization method that optimizes scales to maintain high accuracy at lower bit widths, enabling faster inference with minimal accuracy loss.

Contribution

The paper introduces EasyQuant, a simple and efficient post-training scale optimization technique that achieves near INT8 accuracy at 7 bits, reducing the need for retraining.

Findings

Outperforms TensorRT in accuracy and speed.

Achieves near INT8 accuracy at 7 bits.

Enables faster inference with minimal accuracy loss.

Abstract

The 8 bits quantization has been widely applied to accelerate network inference in various deep learning applications. There are two kinds of quantization methods, training-based quantization and post-training quantization. Training-based approach suffers from a cumbersome training process, while post-training quantization may lead to unacceptable accuracy drop. In this paper, we present an efficient and simple post-training method via scale optimization, named EasyQuant (EQ),that could obtain comparable accuracy with the training-based method.Specifically, we first alternately optimize scales of weights and activations for all layers target at convolutional outputs to further obtain the high quantization precision. Then, we lower down bit width to INT7 both for weights and activations, and adopt INT16 intermediate storage and integer Winograd convolution implementation to accelerate inference.Experimental results on various computer vision tasks show that EQ outperforms the TensorRT method and can achieve near INT8 accuracy in 7 bits width post-training.