Hardware-aware Pruning of DNNs using LFSR-Generated Pseudo-Random Indices

TL;DR

This paper introduces a hardware-aware DNN pruning method using LFSRs to generate non-zero weight locations in real-time, significantly reducing energy and area consumption for embedded applications.

Contribution

The novel approach leverages LFSRs for real-time, hardware-efficient pruning, reducing overhead compared to traditional sparsification techniques.

Findings

Energy savings up to 63.96%

Area reduction up to 64.23%

Effective on VGG-16 with ImageNet data

Abstract

Deep neural networks (DNNs) have been emerged as the state-of-the-art algorithms in broad range of applications. To reduce the memory foot-print of DNNs, in particular for embedded applications, sparsification techniques have been proposed. Unfortunately, these techniques come with a large hardware overhead. In this paper, we present a hardware-aware pruning method where the locations of non-zero weights are derived in real-time from a Linear Feedback Shift Registers (LFSRs). Using the proposed method, we demonstrate a total saving of energy and area up to 63.96% and 64.23% for VGG-16 network on down-sampled ImageNet, respectively for iso-compression-rate and iso-accuracy.