A Quantization-Friendly Separable Convolution for MobileNets

TL;DR

This paper introduces a new quantization-friendly separable convolution architecture for MobileNets, significantly reducing the accuracy gap in 8-bit inference on ImageNet by analyzing and addressing quantization loss.

Contribution

The authors propose a novel separable convolution design that enhances quantization compatibility, improving MobileNetV1's 8-bit inference accuracy close to floating-point performance.

Findings

Achieved 68.03% top-1 accuracy with 8-bit quantization on ImageNet

Identified root causes of quantization loss in MobileNetV1

Demonstrated improved quantization performance with the new architecture

Abstract

As deep learning (DL) is being rapidly pushed to edge computing, researchers invented various ways to make inference computation more efficient on mobile/IoT devices, such as network pruning, parameter compression, and etc. Quantization, as one of the key approaches, can effectively offload GPU, and make it possible to deploy DL on fixed-point pipeline. Unfortunately, not all existing networks design are friendly to quantization. For example, the popular lightweight MobileNetV1, while it successfully reduces parameter size and computation latency with separable convolution, our experiment shows its quantized models have large accuracy gap against its float point models. To resolve this, we analyzed the root cause of quantization loss and proposed a quantization-friendly separable convolution architecture. By evaluating the image classification task on ImageNet2012 dataset, our modified MobileNetV1 model can archive 8-bit inference top-1 accuracy in 68.03%, almost closed the gap to the float pipeline.