Loss-aware Weight Quantization of Deep Networks

TL;DR

This paper introduces a loss-aware weight quantization method for deep networks, extending binarization to ternarization and m-bit quantization, achieving superior compression with minimal accuracy loss.

Contribution

It extends loss-aware weight binarization to ternarization and m-bit quantization, improving compression while maintaining or enhancing accuracy.

Findings

Outperforms state-of-the-art quantization algorithms

Achieves comparable or better accuracy than full-precision networks

Effective on both feedforward and recurrent neural networks

Abstract

The huge size of deep networks hinders their use in small computing devices. In this paper, we consider compressing the network by weight quantization. We extend a recently proposed loss-aware weight binarization scheme to ternarization, with possibly different scaling parameters for the positive and negative weights, and m-bit (where m > 2) quantization. Experiments on feedforward and recurrent neural networks show that the proposed scheme outperforms state-of-the-art weight quantization algorithms, and is as accurate (or even more accurate) than the full-precision network.