Structure-Aware Automatic Channel Pruning by Searching with Graph Embedding

TL;DR

This paper introduces a graph convolutional network-based framework for automatic, structure-aware channel pruning that models global network dependencies to optimize model compression without sacrificing accuracy.

Contribution

It proposes a novel GCN-based method for topology-aware, fully automated channel pruning with a dynamic search over pruning rates, improving over existing heuristics.

Findings

Outperforms state-of-the-art pruning methods in compression efficiency

Maintains competitive accuracy on benchmark datasets

Demonstrates effectiveness across various models like ResNet and VGG16

Abstract

Channel pruning is a powerful technique to reduce the computational overhead of deep neural networks, enabling efficient deployment on resource-constrained devices. However, existing pruning methods often rely on local heuristics or weight-based criteria that fail to capture global structural dependencies within the network, leading to suboptimal pruning decisions and degraded model performance. To address these limitations, we propose a novel structure-aware automatic channel pruning (SACP) framework that utilizes graph convolutional networks (GCNs) to model the network topology and learn the global importance of each channel. By encoding structural relationships within the network, our approach implements topology-aware pruning and this pruning is fully automated, reducing the need for human intervention. We restrict the pruning rate combinations to a specific space, where the number of combinations can be dynamically adjusted, and use a search-based approach to determine the optimal pruning rate combinations. Extensive experiments on benchmark datasets (CIFAR-10, ImageNet) with various models (ResNet, VGG16) demonstrate that SACP outperforms state-of-the-art pruning methods on compression efficiency and competitive on accuracy retention.