A pruning method based on the dissimilarity of angle among channels and filters

TL;DR

This paper introduces a novel channel pruning method based on the dissimilarity of angles among channels and filters, aiming to reduce computational costs of CNNs while maintaining high accuracy, suitable for resource-limited environments.

Contribution

It proposes a new pruning approach that encodes network similarity and applies angle dissimilarity constraints, improving sparsity and efficiency in CNN compression.

Findings

Reduced 66.86% FLOPs on VGG-16 with 93.31% accuracy.

Achieved 58.46% FLOPs reduction on ResNet-32 with 91.76% accuracy.

Demonstrated effectiveness on CIFAR-10 dataset.

Abstract

Convolutional Neural Network (CNN) is more and more widely used in various fileds, and its computation and memory-demand are also increasing significantly. In order to make it applicable to limited conditions such as embedded application, network compression comes out. Among them, researchers pay more attention to network pruning. In this paper, we encode the convolution network to obtain the similarity of different encoding nodes, and evaluate the connectivity-power among convolutional kernels on the basis of the similarity. Then impose different level of penalty according to different connectivity-power. Meanwhile, we propose Channel Pruning base on the Dissimilarity of Angle (DACP). Firstly, we train a sparse model by GL penalty, and impose an angle dissimilarity constraint on the channels and filters of convolutional network to obtain a more sparse structure. Eventually, the effectiveness of our method is demonstrated in the section of experiment. On CIFAR-10, we reduce 66.86% FLOPs on VGG-16 with 93.31% accuracy after pruning, where FLOPs represents the number of floating-point operations per second of the model. Moreover, on ResNet-32, we reduce FLOPs by 58.46%, which makes the accuracy after pruning reach 91.76%.