A novel channel pruning method for deep neural network compression

TL;DR

This paper introduces a genetic algorithm-based channel pruning method to compress deep CNNs, reducing memory and computation for deployment on resource-limited devices while maintaining accuracy.

Contribution

It proposes a novel channel pruning approach using genetic algorithms combined with layer sensitivity and knowledge distillation, improving efficiency and effectiveness over existing methods.

Findings

Achieves 8x parameter reduction on CIFAR-10 with better accuracy.

Outperforms state-of-the-art pruning methods on CIFAR-100 and ImageNet.

Reduces FLOPs by 3x while maintaining model performance.

Abstract

In recent years, deep neural networks have achieved great success in the field of computer vision. However, it is still a big challenge to deploy these deep models on resource-constrained embedded devices such as mobile robots, smart phones and so on. Therefore, network compression for such platforms is a reasonable solution to reduce memory consumption and computation complexity. In this paper, a novel channel pruning method based on genetic algorithm is proposed to compress very deep Convolution Neural Networks (CNNs). Firstly, a pre-trained CNN model is pruned layer by layer according to the sensitivity of each layer. After that, the pruned model is fine-tuned based on knowledge distillation framework. These two improvements significantly decrease the model redundancy with less accuracy drop. Channel selection is a combinatorial optimization problem that has exponential solution space. In order to accelerate the selection process, the proposed method formulates it as a search problem, which can be solved efficiently by genetic algorithm. Meanwhile, a two-step approximation fitness function is designed to further improve the efficiency of genetic process. The proposed method has been verified on three benchmark datasets with two popular CNN models: VGGNet and ResNet. On the CIFAR-100 and ImageNet datasets, our approach outperforms several state-of-the-art methods. On the CIFAR-10 and SVHN datasets, the pruned VGGNet achieves better performance than the original model with 8 times parameters compression and 3 times FLOPs reduction.