Subtensor Quantization for Mobilenets

TL;DR

This paper investigates quantization challenges in MobileNet architectures, identifies causes of accuracy loss, and proposes solutions that enable near-floating point accuracy with 8-bit post-training quantization.

Contribution

It analyzes quantization issues specific to MobileNet and introduces methods that improve accuracy without per-channel or training-aware adjustments.

Findings

Post-training 8-bit quantization achieves within 0.7% accuracy loss.

Identifies root causes of quantization loss in MobileNet.

Proposes alternative quantization strategies for better performance.

Abstract

Quantization for deep neural networks (DNN) have enabled developers to deploy models with less memory and more efficient low-power inference. However, not all DNN designs are friendly to quantization. For example, the popular Mobilenet architecture has been tuned to reduce parameter size and computational latency with separable depth-wise convolutions, but not all quantization algorithms work well and the accuracy can suffer against its float point versions. In this paper, we analyzed several root causes of quantization loss and proposed alternatives that do not rely on per-channel or training-aware approaches. We evaluate the image classification task on ImageNet dataset, and our post-training quantized 8-bit inference top-1 accuracy in within 0.7% of the floating point version.