CompConv: A Compact Convolution Module for Efficient Feature Learning

TL;DR

CompConv introduces a lightweight, plug-and-play convolution module that significantly reduces computational load and parameters in CNNs while maintaining high performance, enabling more efficient feature learning.

Contribution

The paper proposes the novel CompConv module, a compact convolution operator that can replace standard convolutions to improve efficiency without performance loss.

Findings

CompConv reduces computational cost and parameters significantly.

CompConv maintains comparable accuracy to traditional CNNs.

CompConv outperforms existing compression methods in experiments.

Abstract

Convolutional Neural Networks (CNNs) have achieved remarkable success in various computer vision tasks but rely on tremendous computational cost. To solve this problem, existing approaches either compress well-trained large-scale models or learn lightweight models with carefully designed network structures. In this work, we make a close study of the convolution operator, which is the basic unit used in CNNs, to reduce its computing load. In particular, we propose a compact convolution module, called CompConv, to facilitate efficient feature learning. With the divide-and-conquer strategy, CompConv is able to save a great many computations as well as parameters to produce a certain dimensional feature map. Furthermore, CompConv discreetly integrates the input features into the outputs to efficiently inherit the input information. More importantly, the novel CompConv is a plug-and-play module that can be directly applied to modern CNN structures to replace the vanilla convolution layers without further effort. Extensive experimental results suggest that CompConv can adequately compress the benchmark CNN structures yet barely sacrifice the performance, surpassing other competitors.