Neural Network Pruning by Cooperative Coevolution

TL;DR

This paper introduces CCEP, a cooperative coevolution-based filter pruning method that effectively reduces neural network complexity by pruning filters layer-wise, achieving competitive results with less search space and effort.

Contribution

The paper proposes a novel divide-and-conquer filter pruning algorithm using cooperative coevolution, improving search efficiency and performance over existing evolutionary-based pruning methods.

Findings

CCEP prunes ResNet56 by 63.42% FLOPs on CIFAR10 with 0.24% accuracy drop.

CCEP prunes ResNet50 by 44.56% FLOPs on ImageNet with 0.07% accuracy drop.

The method achieves competitive performance compared to state-of-the-art pruning techniques.

Abstract

Neural network pruning is a popular model compression method which can significantly reduce the computing cost with negligible loss of accuracy. Recently, filters are often pruned directly by designing proper criteria or using auxiliary modules to measure their importance, which, however, requires expertise and trial-and-error. Due to the advantage of automation, pruning by evolutionary algorithms (EAs) has attracted much attention, but the performance is limited for deep neural networks as the search space can be quite large. In this paper, we propose a new filter pruning algorithm CCEP by cooperative coevolution, which prunes the filters in each layer by EAs separately. That is, CCEP reduces the pruning space by a divide-and-conquer strategy. The experiments show that CCEP can achieve a competitive performance with the state-of-the-art pruning methods, e.g., prune ResNet56 for $63.42\%$ FLOPs on CIFAR10 with $-0.24\%$ accuracy drop, and ResNet50 for $44.56\%$ FLOPs on ImageNet with $0.07\%$ accuracy drop.