Channel Compression: Rethinking Information Redundancy among Channels in CNN Architecture

TL;DR

This paper introduces compact convolution, a novel CNN building block that reduces channel redundancy without extra parameters, improving efficiency across multiple multimedia tasks.

Contribution

It proposes a parameter-free channel compression method using compact convolution, combining depth-wise separable and point-wise interchannel operations for efficient feature extraction.

Findings

Achieves effective channel redundancy reduction without additional learnable weights.

Demonstrates high efficiency and effectiveness in multimedia classification tasks.

Enables parallel computation for faster processing.

Abstract

Model compression and acceleration are attracting increasing attentions due to the demand for embedded devices and mobile applications. Research on efficient convolutional neural networks (CNNs) aims at removing feature redundancy by decomposing or optimizing the convolutional calculation. In this work, feature redundancy is assumed to exist among channels in CNN architectures, which provides some leeway to boost calculation efficiency. Aiming at channel compression, a novel convolutional construction named compact convolution is proposed to embrace the progress in spatial convolution, channel grouping and pooling operation. Specifically, the depth-wise separable convolution and the point-wise interchannel operation are utilized to efficiently extract features. Different from the existing channel compression method which usually introduces considerable learnable weights, the proposed compact convolution can reduce feature redundancy with no extra parameters. With the point-wise interchannel operation, compact convolutions implicitly squeeze the channel dimension of feature maps. To explore the rules on reducing channel redundancy in neural networks, the comparison is made among different point-wise interchannel operations. Moreover, compact convolutions are extended to tackle with multiple tasks, such as acoustic scene classification, sound event detection and image classification. The extensive experiments demonstrate that our compact convolution not only exhibits high effectiveness in several multimedia tasks, but also can be efficiently implemented by benefiting from parallel computation.