Prune the Convolutional Neural Networks with Sparse Shrink

TL;DR

This paper introduces the 'Sparse Shrink' algorithm that prunes CNN models by analyzing channel importance through sparse reconstruction, significantly reducing parameters and computation with minimal accuracy loss.

Contribution

The paper presents a novel pruning method for CNNs that effectively reduces model size and computation by analyzing channel importance via sparse reconstruction.

Findings

Reduced 56.77% parameters on CIFAR-100

Cut 73.84% of multiplications with minor accuracy decrease

Demonstrated effectiveness of the 'Sparse Shrink' algorithm

Abstract

Nowadays, it is still difficult to adapt Convolutional Neural Network (CNN) based models for deployment on embedded devices. The heavy computation and large memory footprint of CNN models become the main burden in real application. In this paper, we propose a "Sparse Shrink" algorithm to prune an existing CNN model. By analyzing the importance of each channel via sparse reconstruction, the algorithm is able to prune redundant feature maps accordingly. The resulting pruned model thus directly saves computational resource. We have evaluated our algorithm on CIFAR-100. As shown in our experiments, we can reduce 56.77% parameters and 73.84% multiplication in total with only minor decrease in accuracy. These results have demonstrated the effectiveness of our "Sparse Shrink" algorithm.