Channel Pruning Guided by Spatial and Channel Attention for DNNs in Intelligent Edge Computing

TL;DR

This paper introduces a novel attention-guided channel pruning method for deep neural networks, improving model compression efficiency and accuracy on edge devices by focusing on the most informative features.

Contribution

The paper proposes a new attention module combining spatial and channel attention, and a pruning method guided by this module, enhancing model compression without accuracy loss.

Findings

SCA achieves superior inference accuracy with minimal resource overhead.

CPSCA outperforms existing pruning methods at the same pruning ratios.

Experimental results on benchmark datasets validate the effectiveness of the proposed approach.

Abstract

Deep Neural Networks (DNNs) have achieved remarkable success in many computer vision tasks recently, but the huge number of parameters and the high computation overhead hinder their deployments on resource-constrained edge devices. It is worth noting that channel pruning is an effective approach for compressing DNN models. A critical challenge is to determine which channels are to be removed, so that the model accuracy will not be negatively affected. In this paper, we first propose Spatial and Channel Attention (SCA), a new attention module combining both spatial and channel attention that respectively focuses on "where" and "what" are the most informative parts. Guided by the scale values generated by SCA for measuring channel importance, we further propose a new channel pruning approach called Channel Pruning guided by Spatial and Channel Attention (CPSCA). Experimental results indicate that SCA achieves the best inference accuracy, while incurring negligibly extra resource consumption, compared to other state-of-the-art attention modules. Our evaluation on two benchmark datasets shows that, with the guidance of SCA, our CPSCA approach achieves higher inference accuracy than other state-of-the-art pruning methods under the same pruning ratios.