Learning Efficient Convolutional Networks through Network Slimming

TL;DR

This paper introduces a network slimming technique that prunes channels in CNNs during training, significantly reducing model size and computational cost while maintaining accuracy, suitable for deployment in resource-constrained environments.

Contribution

The paper presents a novel channel-level sparsity method for CNNs that directly applies to modern architectures with minimal overhead, enabling efficient model compression without special hardware.

Findings

20x reduction in model size for VGGNet

5x reduction in computing operations for VGGNet

Effective across multiple CNN architectures

Abstract

The deployment of deep convolutional neural networks (CNNs) in many real world applications is largely hindered by their high computational cost. In this paper, we propose a novel learning scheme for CNNs to simultaneously 1) reduce the model size; 2) decrease the run-time memory footprint; and 3) lower the number of computing operations, without compromising accuracy. This is achieved by enforcing channel-level sparsity in the network in a simple but effective way. Different from many existing approaches, the proposed method directly applies to modern CNN architectures, introduces minimum overhead to the training process, and requires no special software/hardware accelerators for the resulting models. We call our approach network slimming, which takes wide and large networks as input models, but during training insignificant channels are automatically identified and pruned afterwards, yielding thin and compact models with comparable accuracy. We empirically demonstrate the effectiveness of our approach with several state-of-the-art CNN models, including VGGNet, ResNet and DenseNet, on various image classification datasets. For VGGNet, a multi-pass version of network slimming gives a 20x reduction in model size and a 5x reduction in computing operations.