Reliable Identification of Redundant Kernels for Convolutional Neural Network Compression

TL;DR

This paper introduces a layer-wise Ln-norm based pruning criterion for CNNs that more accurately identifies unimportant kernels, leading to effective model compression while maintaining accuracy.

Contribution

It proposes a novel pruning method using output feature map Ln-norms that outperforms existing kernel-norm-based pruning techniques.

Findings

Outperforms existing kernel-norm-based pruning methods.

Achieves a good balance between model size and accuracy.

Demonstrated effectiveness on ImageNet and railway surveillance system.

Abstract

To compress deep convolutional neural networks (CNNs) with large memory footprint and long inference time, this paper proposes a novel pruning criterion using layer-wised Ln-norm of feature maps. Different from existing pruning criteria, which are mainly based on L1-norm of convolution kernels, the proposed method utilizes Ln-norm of output feature maps after non-linear activations, where n is a variable, increasing from 1 at the first convolution layer to inf at the last convolution layer. With the ability of accurately identifying unimportant convolution kernels, the proposed method achieves a good balance between model size and inference accuracy. The experiments on ImageNet and the successful application in railway surveillance system show that the proposed method outperforms existing kernel-norm-based methods and is generally applicable to any deep neural network with convolution operations.