Conditional Automated Channel Pruning for Deep Neural Networks

TL;DR

This paper introduces a Conditional Automated Channel Pruning method that efficiently produces deep neural network models with various compression rates in a single process, outperforming existing methods.

Contribution

The paper presents a novel conditional model that enables multi-rate channel pruning in one step, improving efficiency and performance over traditional fixed-rate methods.

Findings

Models with different compression rates outperform existing methods.

Single pruning process achieves multiple compression rates effectively.

Proposed method reduces computational cost compared to multiple separate pruning runs.

Abstract

Model compression aims to reduce the redundancy of deep networks to obtain compact models. Recently, channel pruning has become one of the predominant compression methods to deploy deep models on resource-constrained devices. Most channel pruning methods often use a fixed compression rate for all the layers of the model, which, however, may not be optimal. To address this issue, given a target compression rate for the whole model, one can search for the optimal compression rate for each layer. Nevertheless, these methods perform channel pruning for a specific target compression rate. When we consider multiple compression rates, they have to repeat the channel pruning process multiple times, which is very inefficient yet unnecessary. To address this issue, we propose a Conditional Automated Channel Pruning(CACP) method to obtain the compressed models with different compression rates through single channel pruning process. To this end, we develop a conditional model that takes an arbitrary compression rate as input and outputs the corresponding compressed model. In the experiments, the resultant models with different compression rates consistently outperform the models compressed by existing methods with a channel pruning process for each target compression rate.