Channel Pruning via Automatic Structure Search

TL;DR

This paper introduces ABCPruner, a novel channel pruning method using the artificial bee colony algorithm to automatically search for optimal network structures, improving efficiency and effectiveness over traditional importance-based pruning techniques.

Contribution

The paper presents a new pruning approach that formulates structure search as an optimization problem solved by ABC, reducing human intervention and enhancing pruning performance.

Findings

More effective pruning results demonstrated on deep networks.

Automatic search reduces human bias and effort.

Efficient end-to-end fine-tuning enabled.

Abstract

Channel pruning is among the predominant approaches to compress deep neural networks. To this end, most existing pruning methods focus on selecting channels (filters) by importance/optimization or regularization based on rule-of-thumb designs, which defects in sub-optimal pruning. In this paper, we propose a new channel pruning method based on artificial bee colony algorithm (ABC), dubbed as ABCPruner, which aims to efficiently find optimal pruned structure, i.e., channel number in each layer, rather than selecting "important" channels as previous works did. To solve the intractably huge combinations of pruned structure for deep networks, we first propose to shrink the combinations where the preserved channels are limited to a specific space, thus the combinations of pruned structure can be significantly reduced. And then, we formulate the search of optimal pruned structure as an optimization problem and integrate the ABC algorithm to solve it in an automatic manner to lessen human interference. ABCPruner has been demonstrated to be more effective, which also enables the fine-tuning to be conducted efficiently in an end-to-end manner. The source codes can be available at https://github.com/lmbxmu/ABCPruner.