Mixed-Precision Inference Quantization: Radically Towards Faster inference speed, Lower Storage requirement, and Lower Loss

TL;DR

This paper introduces a mixed-precision quantization method that achieves lower loss than full precision models, enhancing inference speed and reducing storage, by analyzing noise resilience and layer input effects.

Contribution

The study presents a novel mixed-precision quantization approach that outperforms traditional methods and provides insights into neural network noise resilience and identity mappings.

Findings

Mixed-precision quantization reduces model loss.

Layer input noise significantly impacts quantization loss.

Networks with identity mappings resist quantization effects.

Abstract

Based on the model's resilience to computational noise, model quantization is important for compressing models and improving computing speed. Existing quantization techniques rely heavily on experience and "fine-tuning" skills. In the majority of instances, the quantization model has a larger loss than a full precision model. This study provides a methodology for acquiring a mixed-precise quantization model with a lower loss than the full precision model. In addition, the analysis demonstrates that, throughout the inference process, the loss function is mostly affected by the noise of the layer inputs. In particular, we will demonstrate that neural networks with massive identity mappings are resistant to the quantization method. It is also difficult to improve the performance of these networks using quantization.