Hybrid Pruning: Thinner Sparse Networks for Fast Inference on Edge Devices

TL;DR

This paper proposes hybrid pruning, combining channel and weight pruning, to create thinner, efficient neural networks suitable for edge devices, with a fast sensitivity test to optimize pruning decisions.

Contribution

It introduces a novel hybrid pruning method with a fast sensitivity test for effective channel pruning, enabling resource-efficient deployment on edge devices.

Findings

Significantly improved ResNet50 accuracy on ImageNet with hybrid pruning.

Achieved hardware-friendly channel configurations.

Reduced model size and computation with minimal accuracy loss.

Abstract

We introduce hybrid pruning which combines both coarse-grained channel and fine-grained weight pruning to reduce model size, computation and power demands with no to little loss in accuracy for enabling modern networks deployment on resource-constrained devices, such as always-on security cameras and drones. Additionally, to effectively perform channel pruning, we propose a fast sensitivity test that helps us quickly identify the sensitivity of within and across layers of a network to the output accuracy for target multiplier accumulators (MACs) or accuracy tolerance. Our experiment shows significantly better results on ResNet50 on ImageNet compared to existing work, even with an additional constraint of channels be hardware-friendly number.