Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration

TL;DR

This paper introduces FPGM, a filter pruning method based on geometric median that effectively compresses CNNs by removing redundant filters, improving efficiency without sacrificing accuracy.

Contribution

The paper proposes a novel filter pruning technique using geometric median that overcomes limitations of norm-based criteria, enhancing CNN compression and acceleration.

Findings

FPGM reduces over 52% FLOPs on ResNet-110 with accuracy gain on CIFAR-10.

FPGM cuts more than 42% FLOPs on ResNet-101 without accuracy loss on ImageNet.

The method outperforms previous filter pruning approaches in efficiency and effectiveness.

Abstract

Previous works utilized ''smaller-norm-less-important'' criterion to prune filters with smaller norm values in a convolutional neural network. In this paper, we analyze this norm-based criterion and point out that its effectiveness depends on two requirements that are not always met: (1) the norm deviation of the filters should be large; (2) the minimum norm of the filters should be small. To solve this problem, we propose a novel filter pruning method, namely Filter Pruning via Geometric Median (FPGM), to compress the model regardless of those two requirements. Unlike previous methods, FPGM compresses CNN models by pruning filters with redundancy, rather than those with ''relatively less'' importance. When applied to two image classification benchmarks, our method validates its usefulness and strengths. Notably, on CIFAR-10, FPGM reduces more than 52% FLOPs on ResNet-110 with even 2.69% relative accuracy improvement. Moreover, on ILSVRC-2012, FPGM reduces more than 42% FLOPs on ResNet-101 without top-5 accuracy drop, which has advanced the state-of-the-art. Code is publicly available on GitHub: https://github.com/he-y/filter-pruning-geometric-median