On the Acceleration of Deep Neural Network Inference using Quantized Compressed Sensing

TL;DR

This paper introduces a novel binary quantization method based on quantized compressed sensing to accelerate DNN inference on resource-limited devices, aiming to reduce accuracy loss while improving speed and memory efficiency.

Contribution

It proposes a new binary quantization function leveraging quantized compressed sensing, theoretically reducing quantization error compared to standard methods.

Findings

The proposed QCS-based quantization reduces quantization error.

The method maintains the benefits of standard quantization techniques.

Theoretical analysis supports improved accuracy preservation.

Abstract

Accelerating deep neural network (DNN) inference on resource-limited devices is one of the most important barriers to ensuring a wider and more inclusive adoption. To alleviate this, DNN binary quantization for faster convolution and memory savings is one of the most promising strategies despite its serious drop in accuracy. The present paper therefore proposes a novel binary quantization function based on quantized compressed sensing (QCS). Theoretical arguments conjecture that our proposal preserves the practical benefits of standard methods, while reducing the quantization error and the resulting drop in accuracy.