On the impact of the parametrization of deep convolutional neural networks on post-training quantization

TL;DR

This paper develops new theoretical bounds for the output approximation error of quantized CNNs, showing significant improvements over existing results, especially for deep networks like MobileNetV2 and ResNets.

Contribution

It introduces novel approximation bounds for quantized CNNs that significantly outperform previous bounds, especially considering layerwise parametrization and depth effects.

Findings

Bounds improve with network depth, reducing quantization error.

Numerical experiments validate the theoretical bounds on MobileNetV2 and ResNets.

Significant order-of-magnitude gains over state-of-the-art results.

Abstract

This paper introduces novel theoretical approximation bounds for the output of quantized neural networks, with a focus on convolutional neural networks (CNN). By considering layerwise parametrization and focusing on the quantization of weights, we provide bounds that gain several orders of magnitude compared to state-of-the-art results on classical deep convolutional neural networks such as MobileNetV2 or ResNets. These gains are achieved by improving the behaviour of the approximation bounds with respect to the depth parameter, which has the most impact on the approximation error induced by quantization. To complement our theoretical result, we provide a numerical exploration of our bounds on MobileNetV2 and ResNets.