Sub-network Multi-objective Evolutionary Algorithm for Filter Pruning

TL;DR

This paper introduces SMOEA, a multi-objective evolutionary algorithm that efficiently prunes filters in deep neural networks by formulating a sub-network based optimization, leading to lightweight models with minimal accuracy loss.

Contribution

The paper proposes a novel SMOEA method that formulates filter pruning as a multi-objective optimization problem on sub-networks, improving efficiency and performance.

Findings

SMOEA achieves a pruned VGG-14 model with only 0.28% accuracy decrease.

The method reduces parameters by 16.56%, outperforming traditional criteria.

Experiments validate the effectiveness of SMOEA on CIFAR-10.

Abstract

Filter pruning is a common method to achieve model compression and acceleration in deep neural networks (DNNs).Some research regarded filter pruning as a combinatorial optimization problem and thus used evolutionary algorithms (EA) to prune filters of DNNs. However, it is difficult to find a satisfactory compromise solution in a reasonable time due to the complexity of solution space searching. To solve this problem, we first formulate a multi-objective optimization problem based on a sub-network of the full model and propose a Sub-network Multiobjective Evolutionary Algorithm (SMOEA) for filter pruning. By progressively pruning the convolutional layers in groups, SMOEA can obtain a lightweight pruned result with better performance.Experiments on VGG-14 model for CIFAR-10 verify the effectiveness of the proposed SMOEA. Specifically, the accuracy of the pruned model with 16.56% parameters decreases by 0.28% only, which is better than the widely used popular filter pruning criteria.