Pruning Ternary Quantization

TL;DR

This paper introduces pruning ternary quantization (PTQ), a novel method that effectively compresses deep neural networks by integrating normalization, pruning, and weight decay to produce highly accurate, low-bit ternary models with significant size reduction.

Contribution

The paper proposes a new symmetric ternary quantization method that simultaneously optimizes bit-width, model size, and accuracy, outperforming existing approaches in compression and accuracy.

Findings

Achieves 49× compression of ResNet-18 with only 4% accuracy drop.

Compresses Mask R-CNN from 170MB to 5MB with 2.8% AP drop.

Validated on various networks and tasks, demonstrating broad applicability.

Abstract

Inference time, model size, and accuracy are three key factors in deep model compression. Most of the existing work addresses these three key factors separately as it is difficult to optimize them all at the same time. For example, low-bit quantization aims at obtaining a faster model; weight sharing quantization aims at improving compression ratio and accuracy; and mixed-precision quantization aims at balancing accuracy and inference time. To simultaneously optimize bit-width, model size, and accuracy, we propose pruning ternary quantization (PTQ): a simple, effective, symmetric ternary quantization method. We integrate L2 normalization, pruning, and the weight decay term to reduce the weight discrepancy in the gradient estimator during quantization, thus producing highly compressed ternary weights. Our method brings the highest test accuracy and the highest compression ratio. For example, it produces a 939kb (49$\times$) 2bit ternary ResNet-18 model with only 4\% accuracy drop on the ImageNet dataset. It compresses 170MB Mask R-CNN to 5MB (34$\times$) with only 2.8\% average precision drop. Our method is verified on image classification, object detection/segmentation tasks with different network structures such as ResNet-18, ResNet-50, and MobileNetV2.