Quantization in Layer's Input is Matter

TL;DR

This paper demonstrates that quantizing layer inputs has a greater impact on loss function performance than quantizing parameters, and proposes an input-based quantization algorithm outperforming Hessian-based methods.

Contribution

It introduces a novel input-based quantization algorithm that surpasses Hessian-based mixed precision approaches in neural network quantization.

Findings

Input quantization significantly affects loss function more than parameter quantization.

The proposed algorithm outperforms Hessian-based mixed precision methods.

Layer input quantization leads to better model performance.

Abstract

In this paper, we will show that the quantization in layer's input is more important than parameters' quantization for loss function. And the algorithm which is based on the layer's input quantization error is better than hessian-based mixed precision layout algorithm.