# Focused Quantization for Sparse CNNs

## TL;DR

This paper introduces focused quantization, a power-of-two based strategy for sparse CNNs that significantly reduces model size and computational cost while maintaining high accuracy, enabling efficient deployment on constrained devices.

## Contribution

The paper proposes a novel focused quantization method that dynamically adapts to weight distributions in sparse CNNs, replacing multiplications with bit-shifts for efficient inference.

## Key findings

- Achieved 18.08x compression ratio on ResNet-50 with only 0.24% accuracy loss.
- Fully compressed ResNet-18 with higher CR and accuracy than existing methods.
- Reduced hardware complexity by requiring fewer logic gates for implementation.

## Abstract

Deep convolutional neural networks (CNNs) are powerful tools for a wide range of vision tasks, but the enormous amount of memory and compute resources required by CNNs pose a challenge in deploying them on constrained devices. Existing compression techniques, while excelling at reducing model sizes, struggle to be computationally friendly. In this paper, we attend to the statistical properties of sparse CNNs and present focused quantization, a novel quantization strategy based on power-of-two values, which exploits the weight distributions after fine-grained pruning. The proposed method dynamically discovers the most effective numerical representation for weights in layers with varying sparsities, significantly reducing model sizes. Multiplications in quantized CNNs are replaced with much cheaper bit-shift operations for efficient inference. Coupled with lossless encoding, we built a compression pipeline that provides CNNs with high compression ratios (CR), low computation cost and minimal loss in accuracy. In ResNet-50, we achieved a 18.08x CR with only 0.24% loss in top-5 accuracy, outperforming existing compression methods. We fully compressed a ResNet-18 and found that it is not only higher in CR and top-5 accuracy, but also more hardware efficient as it requires fewer logic gates to implement when compared to other state-of-the-art quantization methods assuming the same throughput.