Exploring the Effects of Channel Sparsity on Neural Network Pruning for Acoustic Scene Classification

TL;DR

This paper investigates how channel sparsity affects neural network pruning in Acoustic Scene Classification, introducing a new metric and comparing pruning methods to optimize model efficiency without sacrificing accuracy.

Contribution

It introduces Weight Skewness as a novel metric for channel sparsity and provides a comprehensive analysis of pruning effects across different neural network architectures.

Findings

Higher channel sparsity leads to greater compression with acceptable accuracy loss.

Pruning method choice has minimal impact on compression results.

MobileNets benefit more from sparsification than VGGNets and ResNets.

Abstract

Acoustic Scene Classification (ASC) algorithms are usually expected to be deployed in resource-constrained systems. Existing works reduce the complexity of ASC algorithms by pruning some components, e.g. pruning channels in neural network. In practice, neural networks are often trained with sparsification such that unimportant channels can be found and further pruned. However, little efforts have been made to explore the the impact of channel sparsity on neural network pruning. To fully utilize the benefits of pruning for ASC, and to make sure the model performs consistently, we need a more profound comprehension of channel sparsification and its effects. This paper examines the internal weights acquired by convolutional neural networks that will undergone pruning. The study discusses how these weights can be utilized to create a novel metric, Weight Skewness (WS), for quantifying the sparsity of channels. We also provide a new approach to compare the performance of different pruning methods, which balances the trade-off between accuracy and complexity. The experiment results demonstrate that 1) applying higher channel sparsity to models can achieve greater compression rates while maintaining acceptable levels of accuracy; 2) the selection of pruning method has little influence on result 1); 3) MobileNets exhibit more significant benefits from channel sparsification than VGGNets and ResNets.