Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle

TL;DR

This paper introduces a structured sparsification method that automatically induces sparsity in convolutional weights and employs a learnable channel shuffle to efficiently compress CNNs with minimal performance loss.

Contribution

It presents a novel joint optimization approach for structured sparsity and channel shuffling, improving network compression efficiency and accuracy.

Findings

Achieves competitive accuracy with reduced computational complexity.

Effectively induces structured sparsity for efficient implementation.

Maintains performance with negligible accuracy drop across various architectures.

Abstract

Recent advances in convolutional neural networks(CNNs) usually come with the expense of excessive computational overhead and memory footprint. Network compression aims to alleviate this issue by training compact models with comparable performance. However, existing compression techniques either entail dedicated expert design or compromise with a moderate performance drop. In this paper, we propose a novel structured sparsification method for efficient network compression. The proposed method automatically induces structured sparsity on the convolutional weights, thereby facilitating the implementation of the compressed model with the highly-optimized group convolution. We further address the problem of inter-group communication with a learnable channel shuffle mechanism. The proposed approach can be easily applied to compress many network architectures with a negligible performance drop. Extensive experimental results and analysis demonstrate that our approach gives a competitive performance against the recent network compression counterparts with a sound accuracy-complexity trade-off.