Dynamic Shuffle: An Efficient Channel Mixture Method

TL;DR

This paper introduces a dynamic shuffle module that generates data-dependent channel permutation matrices to improve neural network efficiency and performance with minimal extra computation, validated on multiple image classification datasets.

Contribution

The paper proposes a novel dynamic shuffle method that adaptively mixes channels using learnable, data-dependent permutation matrices, enhancing CNN performance and efficiency.

Findings

Significant performance improvement on CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet.

Efficient generation of permutation matrices with negligible extra computation.

Static-dynamic-shuffle serves as a lightweight replacement for pointwise convolution.

Abstract

The redundancy of Convolutional neural networks not only depends on weights but also depends on inputs. Shuffling is an efficient operation for mixing channel information but the shuffle order is usually pre-defined. To reduce the data-dependent redundancy, we devise a dynamic shuffle module to generate data-dependent permutation matrices for shuffling. Since the dimension of permutation matrix is proportional to the square of the number of input channels, to make the generation process efficiently, we divide the channels into groups and generate two shared small permutation matrices for each group, and utilize Kronecker product and cross group shuffle to obtain the final permutation matrices. To make the generation process learnable, based on theoretical analysis, softmax, orthogonal regularization, and binarization are employed to asymptotically approximate the permutation matrix. Dynamic shuffle adaptively mixes channel information with negligible extra computation and memory occupancy. Experiment results on image classification benchmark datasets CIFAR-10, CIFAR-100, Tiny ImageNet and ImageNet have shown that our method significantly increases ShuffleNets' performance. Adding dynamic generated matrix with learnable static matrix, we further propose static-dynamic-shuffle and show that it can serve as a lightweight replacement of ordinary pointwise convolution.