Feature Statistics Guided Efficient Filter Pruning

TL;DR

This paper introduces a novel filter pruning method for CNNs that leverages feature map diversity and similarity to effectively reduce model size and computational cost without sacrificing accuracy.

Contribution

It proposes a new pruning approach using diversity-aware and similarity-aware feature map selection, improving efficiency over traditional methods.

Findings

Achieves up to 91.6% parameter reduction

Reduces FLOPs by 83.7%

Maintains accuracy with significant model compression

Abstract

Building compact convolutional neural networks (CNNs) with reliable performance is a critical but challenging task, especially when deploying them in real-world applications. As a common approach to reduce the size of CNNs, pruning methods delete part of the CNN filters according to some metrics such as $l1$-norm. However, previous methods hardly leverage the information variance in a single feature map and the similarity characteristics among feature maps. In this paper, we propose a novel filter pruning method, which incorporates two kinds of feature map selections: diversity-aware selection (DFS) and similarity-aware selection (SFS). DFS aims to discover features with low information diversity while SFS removes features that have high similarities with others. We conduct extensive empirical experiments with various CNN architectures on publicly available datasets. The experimental results demonstrate that our model obtains up to 91.6% parameter decrease and 83.7% FLOPs reduction with almost no accuracy loss.