GoPrune: Accelerated Structured Pruning with $\ell_{2,p}$-Norm Optimization

TL;DR

GoPrune introduces an accelerated structured pruning technique using $\\ell_{2,p}$-norm optimization and an efficient PAM algorithm, significantly improving CNN compression and inference speed on resource-limited devices.

Contribution

It extends the $\ell_p$-norm to $\ell_{2,p}$-norm for structured pruning and develops a closed-form solution-based optimization algorithm, enhancing pruning efficiency.

Findings

Outperforms existing pruning methods on CIFAR with ResNet and VGG.

Achieves higher compression ratios with maintained accuracy.

Demonstrates faster convergence and better efficiency.

Abstract

Convolutional neural networks (CNNs) suffer from rapidly increasing storage and computational costs as their depth grows, which severely hinders their deployment on resource-constrained edge devices. Pruning is a practical approach for network compression, among which structured pruning is the most effective for inference acceleration. Although existing work has applied the $\ell_p$-norm to pruning, it only considers unstructured pruning with $p\in (0, 1)$ and has low computational efficiency. To overcome these limitations, we propose an accelerated structured pruning method called GoPrune. Our method employs the $\ell_{2,p}$-norm for sparse network learning, where the value of $p$ is extended to $[0, 1)$. Moreover, we develop an efficient optimization algorithm based on the proximal alternating minimization (PAM), and the resulting subproblems enjoy closed-form solutions, thus improving compression efficiency. Experiments on the CIFAR datasets using ResNet and VGG models demonstrate the superior performance of the proposed method in network pruning. Our code is available at https://github.com/xianchaoxiu/GoPrune.