Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Coding

TL;DR

This paper introduces a novel post-training neural network compression method combining rate-aware quantization and entropy coding, achieving significant size reduction while maintaining performance, suitable for resource-constrained devices.

Contribution

The authors propose a new compression framework that extends layer-wise loss with quadratic rate estimation and uses locally exact solutions via OBS, enabling fast decoding and compatibility with arbitrary quantization grids.

Findings

Achieved 20-40% bit rate reduction with maintained performance.

Outperformed popular compression algorithm NNCodec in experiments.

Validated on various computer-vision networks.

Abstract

The ever-growing size of neural networks poses serious challenges on resource-constrained devices, such as embedded sensors. Compression algorithms that reduce their size can mitigate these problems, provided that model performance stays close to the original. We propose a novel post-training compression framework that combines rate-aware quantization with entropy coding by (1) extending the well-known layer-wise loss by a quadratic rate estimation, and (2) providing locally exact solutions to this modified objective following the Optimal Brain Surgeon (OBS) method. Our method allows for very fast decoding and is compatible with arbitrary quantization grids. We verify our results empirically by testing on various computer-vision networks, achieving a 20-40\% decrease in bit rate at the same performance as the popular compression algorithm NNCodec. Our code is available at https://github.com/Conzel/cerwu.